As part of my research for the Institute of BackYard Studies , I am prototyping various components for a Lightly Oscillating Linguistic Organ (LoLa) – a musical instrument that harnesses the sinusoidal beauty of the pendulum wave. Inspired by a creative brain fart described in “Nightmare on IBYS street”, I set about prototyping the playing of sounds from android phone sensors.
After much googling around and substantial fiddling, here is what I did:
- Searched around for quick and dirty ways to create an android app using PROCESSING to play sounds based on sensor input. Found out pretty quickly there are many can-o-worms and significant issues around latency.
- Decided to simply mash up a few sensor / music apps for a V1.0 prototype.
- The AndroSenso app simply outputs raw data from each sensor and can make graph / log file recording for a period of time. After looking intently at accelerometer data I decided the Proximity Sensor would be a better option for my first prototype sound trigger.
- The Proximity Actions app allows actions to be configured when various numbers of ‘waves’ past the proximity sensor are detected. It is pretty well thought out, behaves well in the background and can easily trigger music play / pause.
Here is what I learnt:
- Latency, latency, latency – such a long wait before any sound starts. Varies depending on the state the phone is in, gets worse over time…
- APPS: accelerometer sensor, physics Toolbox, androSensor (the most useful), proximity sensor finder, proximity sensor clib, sensor music player, music proximity, proximity actions (the most useful).
- Good as it is, Proximity Actions has it’s limitations & points to a whole lot of pain in developing my own specific trigger app. A minimum of two ‘waves’ must be detected before and action can be triggered. The proximity sensor on the HUAWEI ascend g510 running android v4.1.1 is soooo frustrating. I could only get it to detect two waves with Max ave time at .4 sec & max wave timeout at .7 sec. And then it is frustrating as hell to reliably produce the two waves required to trigger the music.
- And then there is the LATENCY. Not only is it horrible but it varies. I’m guessing the variation is due to the phone o/s swapping out or shutting down things like the music player after a while. Nearly a second delay at best, several seconds when things go pear shaped.
- If it is crap as a prototype, things will probably get worse trying to hard code this and scale up to more than 8 devices swinging at once! My guess is that proximity sensors (and indeed others) would vary from phone to phone – making consistency difficult. From everything I read, latency is always going to be a problem. Finally, the phone is designed to provide a good phone experience – there are a lot of environment considerations to take into account when building an app to quickly trigger sounds, day in, day out..