open sound control

Real-time Gesture Control for Audio Signals

Summary

Open Sound Control (OSC) is a network protocol invented to replace MIDI for modern computer-based musical control systems. OSC was created by an interdisciplinary team at CNMAT, UC Berkeley, and has gained widespread adoption in the electronic music and media production community.

One of the lesser-used features of OSC is the ability to incorporate timestamps in OSC Bundles. Timestamps are essential in gesture streaming to remove temporal jitter noise caused by transportation over best-effort network protocols and non-real-time scheduling in desktop operating systems.

While at CNMAT, I extended the OSC tools for Max/MSP to enable correct event scheduling of timestamp-encoded OSC data streams, including down to the sample level in real-time audio systems. I also implemented an OSC-based clock synchronization protocol to enable timestamp-based jitter reduction between a USB-connected PIC microcontroller and the host computer.

I also worked on a concept for storing and retrieving OSC-encoded gesture data using an SQL database capable of recording and replaying timestamp-encoded OSC data. A challenge of using SQL for this application is optimizing the index types to support a combination of OSC query matching (XPath-like matching) and selection based on continuous data ranges.

Skills

Real-time Event Scheduling Algorithms
Signal Processing
Pattern Matching Languages
SQL, Index Optimization
C Programming Language
Max/MSP
Microcontrollers

References

Freed, Adrian, John MacCallum, and Andrew Schmeder. "Composability for Musical Gesture Signal Processing using new OSC-based Object and Functional Programming Extensions to Max/MSP." NIME. 2011.

Freed, Adrian, John MacCallum, and Andrew Schmeder. "Dynamic, Instance-based, object-oriented programming in Max/MSP using open sound control message delegation." ICMC. 2011.

Schmeder, A., A. Freed, and D. Wessel. "Best Practices for Open Sound Control." Linux Audio Conference, Utrecht, NL (2010).

Freed, A., and A. Schmeder. "Features and Future of Open Sound Control version 1. 1." Proceedings of the 2009 New Interfaces for Musical Expression (NIME) Conference. 2009.

Schmeder, Andrew, and Adrian Freed. "A Low-level Embedded Service Architecture for Rapid DIY Design of Real-time Musical Instruments." NIME. 2009.

Schmeder, Andrew. "Efficient gesture storage and retrieval for multiple applications using a relational data model of open sound control." ICMC. 2009.

Schmeder, Andrew, and Adrian Freed. "uOSC: The Open Sound Control Reference Platform for Embedded Devices." NIME. 2008.

Schmeder, Andy, and Adrian Freed. "Implementation and Applications of Open Sound Control Timestamps." ICMC. 2008.

Schmeder, Andy. "Everything you ever wanted to know about Open Sound Control." Center for New Music and Audio Technologies, University of California, Berkeley, Tech. Rep (2008).

Schmeder, Andrew, et al. "CNMAT Information Infrastructure." ICMC. 2007.

Freed, Adrian, Andy Schmeder, and Michael Zbyszynski. "Open Sound Control--A flexible protocol for sensor networking." (Long. Beach. Calif) (2007).

Schmeder, A. W., and M. Wright. "A query system for Open Sound Control." Draft Proposal (2004).

Page updated

Google Sites

Report abuse