For the Amsterdam Museum Night 2018 me and the research group at STEIM (Daniele Fabris, Fedde Ten Berge, Andreas Tegnander) created motion-based wireless controllers for the Pentacle System 15.3.

With these controllers, the audience can play electronic music themselves and experience it in a spatial sound configuration. The users can play with a total of three controllers, each one with a different musical role and audio generation system. The controllers contain a orientation sensor that allow the users to generate and manipulate sound trough their movements. An LED ring built in the controllers allows further feedback on the user actions.
The controllers are based on the ESP32 microcontroller and LSM9SD1 IMU sensor.
My task was to develop the firmware for the microcontroller to have a bidirectional communication between the controllers and a main computer: to read the data from the sensors and to send LED light information back to the controllers.
I also implemented some orientation processing techniques in MAX and Javascript to “cook” the data into easy to use parameters for audio manipulation.

credits:
Controller concept/design: Fedde ten Berge
Electronics and firmware: Michele Abolaffio / Fedde ten Berge
Software for rhythms: Andreas Tegnander
Software for melodies: Daniele Fabris
Software for chaotic oscillators and sferical drones: Fedde ten Berge

 

Advertisements