I.C.E. Kinect DAW Controller
I.C.E. = Interactive Control Environment is a DAW Controller (Digital Audio Workstation Controller). It uses Kinect for hand-tracking. It uses Quartz Composer and Obj-C as its core. It supports both OSC and MIDI. Intended Usage : Kinect + Augmented Reality Glasses (I used a normal projector for demonstration purposes).
Possible Usage Fields
- Recording Studio, DAW – Recording
- Broadcast Studio, Signal Routing
- Concert Audio, Live Mixing
- Concert Lights, DMX / CUE Control
Functions List / How to use ICE
Left Hand – Channel Selected
- Right hand Y Axis = Volume
- Right hand X Axis = Pan
- Right hand Z Axis = Reverb or Comp (Assignable & Only works in ICE Live)
- Rec Arm, Input Monitoring, Solo, Mute, EQ, Compressor, Reverb = Activated & Deactivated with right hand.
Left/Right Hand Modifier
- Level Protection (L/R Hand Modifier) (ex :
- Group Clutch (L/R Hand Modifier) (ex :
- Right Hand False Activity Protection (L/R Modifier) (ex :
- EXIT button also works as an arm-rest. so if your arms are tired, and there is nothing else you need to do, holding your hands on the EXIT button, one by one, will stop tracking them.
- Holding both of your hands on EXIT, mutes all channels. (emergency feedback protection)
Evolution of ICE
The first design had the transport bar on the top. I don’t usually go back in the project that often, so I thought placing the transport bar higher wouldn’t be a big problem. I placed the channels on the left, and spread the function buttons. The biggest problem was the “click” we’re all used to from the mouse. I don’t like clicking because I felt it’s not natural. In nature, you can’t just click and watch things happen, so I spent a lot of time thinking how I can use both hands, and reduce or even remove the “click”. I utilized a modifier-key system. When the left hand is over channel 1, the right hand’s Y adjusts the volume and X adjusts panning. (A new experimental feature : Z adjusts the reverb) It feels extremely natural, because it’s lifelike. It’s just like you’re holding the source and moving left-right or moving it away from you so you start to hear the reverb/delay and the room effect.
I used the first prototype for a week to note down what worked and what did not. I found myself escaping from the transport bar, because it’s harder to reach. Channels did feel right, but because of the general sensitivity it was hard to hold my hand over a channel while I’m adjusting something with my right hand. I changed the entire layout.
I removed the top bar, used a different channel-layout, made the function buttons a little bit smaller, and gained some space for the “concert feed” (possible future implementation, using AR-Glasses). Now, I don’t have to reach up to the top bar and the transport buttons are in a less-tiring place. Channels 5/13 – 6/14 – 7/15 – 8/16 are not that high now, so I have happier left arm muscles now. I realized, I didn’t actually use the Bus Channels that much in the first rough/raw record – I removed the bus channels.
The second design feels natural, fast, and overall more efficient than the first one. After using the first two prototypes a few times, ideas flooded my brain. I noticed that ICE feels better this way. Unlike a normal desktop DAW Controller, ICE didn’t require any special knowledge because it feels incredibly natural without ‘clicking’, and I’m physically involved; controlling the rough mix by pushing the sources away, or moving them left-right; focusing on the sound rather than focusing on faders/potentiometers. I can hear small details better with ICE than mixing with normal desktop controllers where I felt like a technician, left out, not a part of the mix. However, mixing with ICE, I feel like I’m a conductor, and I’m a part of the record/mix.
Other Possible Implementation Fields
- This system can also be used in concerts. With AR-Glasses, you can select the player with your left hand, and adjust the volume/pan/reverb with your right hand.
- This system can also be used as a light-cue controller/DMX in concerts or other events. With your left hand, you can select light-groups, and with your right hand you can turn them on/off or adjust their brightness. Even more, you can control a big intelligent light rig, like a DMX. You can select a light with your left hand, and move it with your right hand.