Introducing Space Palette...

Using the unique features of Resolume, FFGL plugins, and the three-dimensional input capability of the Kinect, Tim Thompson has created the Space Palette. It's an instrument that lets you perform visuals and music controlled directly from the motion of your hands, using no pre-recorded media.

In the current Space Palette, each of four large holes lets you play four different sounds and draw four different types of graphics, all simultaneously using any number of hands or objects. The smaller holes act as buttons that let you change sounds and colors, turn on looping, control musical keys and scales, etc. Although originally and still primarily designed as a "casual instrument" installation for events like Burning Man, Tim occasionally does performances - here are some recent ones:



The software behind the Space Palette has two parts. The first part is a standalone program (called MultiMultiTouchTouch or MMTT) that lets you use the Kinect to define any number of three-dimensional multitouch surfaces in mid-air; it can be trained within a minute on any flat frame with holes. Each hole essentially becomes a multitouch mousepad with depth, in which any number of hands (or objects) can be used simultaneously.

The output of MMTT is TUIO, a standard for conveying multitouch information over OSC. TUIO fortunately includes a way of describing three-dimensional cursors, and MMTT and the Space Palette make good use of it. The source code for MMTT has been made freely available to the Kinect hacker community.

The second part of the Space Palette is an FFGL plugin that Tim wrote called Space Manifold, which runs inside Resolume. This plugin takes the TUIO output of MMTT, and directly generates both visuals and music. Both types of output are controlled directly by your hands; there are no pre-recorded clips. The visuals are generated using OpenGL, and musical notes are generated by sending MIDI over to soft synthesizers.

Tim currently uses Plogue Bidule as a VST host, and Alchemy for most of the sounds. The visuals done by the Space Manifold plugin are further enhanced by a chain of 13 other FFGL plugins that are provided with Resolume. These effects are selectively enabled and disabled under the control of the Manifold plugin (more details below).

Both MMTT and the Space Manifold plugin use browser-based interfaces for control. This means that any browser (including one on a different computer or tablet) can be used to control them.

One of the more interesting aspects of the Space Manifold FFGL plugin is that it makes excellent use of Resolume's ability to be controlled by OSC. When Manifold decides it wants to enable or disable some of the other effects (due to gestures detected within the Space Palette), it sends OSC messages to Resolume to do so. This demonstrates that an FFGL plugin can actually control its own host, in ways not anticipated by the FFGL standard. Resolume itself doesn't even realize that the OSC it is receiving is coming from one of the FFGL plugins that it is hosting!

Additional design intent and technical details can be found in these slides from a talk, Tim Thompson's Home Page and the Space Palette page. Tim has plans for providing a Python-based interface for FFGL plugins, making it easier to create and change behaviours (both musical and visual) on the fly, without restarting the FFGL host. Anyone interested in that avenue of exploration is encouraged to contact Tim via me@timthompson.com