Project 4 (Interact)
Here are some libraries/environments that might be useful for this project:
Kinect
The Microsoft Kinect is a cheap, commercial depth sensing camera. Yu can get both a RGB image and a 3D depth map out of it. Some libraries can use this depth map to track people via abstracting them into simplified skeletons.
There are currently 3 main libraries for working with with the kinect:
- libfreenect, reverse-engineered open source C library
- OpenNI, PrimeSense library with skeleton tracking
- Microsoft Kinect for Windows
There are wrappers around some of these libs for most creative coding environments. Here are a few:
Processing
- Daniel Shiffman’s Kinect in Processing
- simple-openni
OpenFrameworks
- ofxKinect: uses open source freenect library, simple to use, no skeleton tracking, works on Mac/Win/Linux
- ofxOpenNI: uses PrimeSense OpenNI library, more difficult to use, has skeleton tracking, may not work, YMMV
Cinder
Max
There are also a few pre-built Kinect apps that can stream skeleton data over OSC, so you don’t have to work with it directly:
- Synapse Win/Mac, I made a Processing template you might use
- OSCeleton Win/Max/linux
AR Tool Kit
The Augmented Reality toolkit is a library for finding and tracking image markers as well as providing a 3d transform for drawing content to fit over the marker. You will need to use the markers provided with the libraries.
Processing
OpenFrameworks
- ofxARToolkitPlus, Mac only
Cinder
Max
Reactivision
Reactivision is the fiducial marker tracking application that streams the marker info using the TUIO protocol via OSC. It was developed for the ReacTable project and is 2D only since it’s optimized for use with an IR camera and projector mounted underneath the projection surface. The markers are then attached to objects and placed facing down on the projection surface.
See the Reactivision page for the app and plenty of examples and templates for Processing ,OF, Max, Pure Data, etc.