SoniZen -  Data Sonification @Columbia Computer Music Center
🧘🏻

SoniZen - Data Sonification @Columbia Computer Music Center

Created
Mar 21, 2022 09:17 PM
update
Last updated November 26, 2022
Domain
Music/Audio Technology
time
2022
💡
May 20, 2022 A multi-modal meditative music patch in MAX/MSP and Ableton Live.
Listen to the sound of your body and make sound happen by being.
Instructed by Professor Brad Garton.

Notion:

motion(gesture) + body signals + environmental signals = music
When we do meditations, we often close our eyes, stop our movements and focus on our breath, and any changes in our body. Often, they are too subtle to detect, especially for people in a busy world. What if we re-activate our body through sound, close our senses and let our body and environment around us do the job?

Tools

  1. Max/MSP for data processing and plug-in design.
  1. Fluid Corpus Manipulation project(FluCoMa) for neural network module.
  1. Ableton Live for multi-modal live performing or music making.
  1. Holon.ist for accessing data generated by iPhone, AirPods, Apple Watch.

Data Source & Mapping

  1. motion (gestures)
    1. breath measured by subtle movement of airpods attitude and rotation
      1. notion image
  1. body signals: heart beat from apple watch as tempo (BPM)
    1. notion image
  1. environmental signals:
    1. local sun/moon elevation, wind speed, temperature, humidity, time from online data
      1. notion image
    2. brightness from camera
      1. materials from jitter image processing and motion detection
        notion image
    3. environmental sound from microphone
      1. make a live sound event classifier based on mubu.gmm.scratch, then re-synthsesize the sound
        a small demo shows how this would work
       

Max-for-Live Plugin Design

mapping signals to sound processing parameters
notion image
notion image
notion image

A little demo for the Max for Live patch

a small demo shows the training process and then how parameters start to change to output different sound when I put on my AirPods and started moving my head

Codes

SoniZen
sivannavis • Updated Nov 21, 2022