(apologies for cross-posting)

I am very happy to announce the Open Source release of PESI extended system developed at SOPI research group. This system is designed for co-located collaboration, providing spatial opportunities for musical exploration.

The software system includes 2 main components;

PESI_OnBody component released as Xcode project to create a mobile instrument as an iOS application with integrated audio synthesis part. This component is a custom software built with Objective-C and Libpd. This release includes a Pure Data patch template with three sound modules to create and develop musical interfaces for iOS mobile phones. PESI_OnBody component uses mobile phone’s sensor input data to create and process sound. All the audio synthesis is done in the device with Libpd. It also streams the sensor data to the _masterController patch in PESI_InSpace component.

PESI_InSpace component (repository includes link for the video tutorial) is a modular multi-user motion tracking system (repositories kinectTrackerRealTime, kinectPlySaver and kinectDataServer), which also provides the data to extract certain features, such as relative distances, velocity, acceleration and alignment. The modular structure of the motion tracking module enables to add several Kinect v1 sensor bars to the system for a reliable tracking. The tracking module avoids the loss of tracked players due to occlusions or abrupt movements. The repository PESI_InSpace, the _masterController patch, receives the tracking data from kinectTrackerRealTime and sensor data from mobile phones. The _masterController patch uses that received data to create / manipulate sound and to distribute it to multi-channel speakers.

This release is one of the research outcomes of The Notion of Participative and Sonic Interaction - Academy of Finland project 137646. The project presented at NIME 13 “PESI Extended System: In Space, On Body, with 3 Musicians”, at SMC 13 “Composing Social Interactions for an Interactive-Spatial Performance System” and “Situating the Performer and the Instrument in a Rich Social Context with PESI Extended System”. The system components have also been used in various music performances.

Github repository

more about the project

best,

Koray