Lecture 11-02-09
Today we are setting up openFrameworks to work with the MS Kinect depth camera.
Install
Grab OF from openframeworks.com/downloads and the following OF Kinect addons:
- ofxKinect: uses open source libfreenect, provides RGB and Depth images (direct download)
- ofxOpenNI: uses close source Primesense OpenNI, provides RGB and Depth images plus a skeleton finder (direct download)
Note: These addons currently only work for Mac and Linux (ofxKinect). We will be working on getting Windows versions shortly.
Uncompressing the openFrameworks zip gives you the OF folder layout:
/addons --- where to place ofx addons /apps --- apps be here /libs /other /scripts
Uncompress ofxKinect and ofxOpenNI and place them in the /addons folder.
Building and Running Apps
You can try any of the example apps in /apps/examples or /example/addonsExamples.
Open the project file in the an example folder with you environment of choice:
- Mac: XCode .xcodeproj
- Linux: Codeblocks .workspace
- Windows: Visual Studio .sln, Codeblocks .workspace
Build the example and run it. Some examples have 2 project files, one for OF 0062 and one for OF 007 beta. Make sure to use the project file for the correct OF version you have.
Note: If you are using XCode and don’t like how it uses multiple windows, I recommend using the All-In-One layout. Set this in Preferences by:
XCode->Preferences->General->Layout->All-In-One
If the Layout dropdown box is grey, close all the XCode windows then open the preferences again.
Creating a New App
The easiest way to create a new app project is to copy an example to your apps folder. The /apps/examples/emptyExample project is a bare bones project which is a good start. Copy the emptyExample folder into /apps/myApps.
To rename to project on Mac:
- rename the project folder
- open the project in XCode and rename it in Projects->Rename
If you want to use an addon, look for one of the addon examples. ofxKinect and ofxOpenNI for instance have example project folders which you can use to start.
Adding an Addon to an Existing App
On Mac:
- in XCode, create a new Group for the addon ie “ofxOpenCV” by right clicking on addons under the source tree and selecting Add->New Group
- drag the src and lib directories from the addon folder in the Finder into the Group you created in XCode
Some external (ofx*) libraries have special instructions such as ofxOpenNI in their README files. Make sure to double check these when using addons which are not included with OF by default.
Here is some simple code that I was using last semester to play with recorded Kinect images.
NOTE: Updated for current ofxKinect. Dan Wilcox
You can record to an iamge sequence and load it later for offline usage. This is very simple, be careful not to record/playback too many! Check your data folder to make sure you don’t fill it too full.
You will need ofxDirList. Code is as follows… (note: not fully tested, treat as psuedo code)
testApp.h:
testApp.cpp:
void testApp::setup() { ofSetFrameRate(30); depthImage.allocate(kinect.width, kinect.height, OF_IMAGE_GRAYSCALE); // using a live kinect? #ifdef KINECT_CONNECTED kinect.init(); kinect.open(); // not connected #else // Load png files from file dir.allowExt("png"); dir.setVerbose(false); string folder = "01/"; totalImages = dir.listDir(folder); saveCounter = 0; currentImage = 0; // load all recorded depth images in "data/01/" for(int i = 0; i < totalImages; i++){ ofImage *img = new ofImage(); img.loadImage(folder + dir.getName(i)); img.setImageType(OF_IMAGE_GRAYSCALE); savedImages.push_back(img); } #endif } void testApp::update() { // load a saved image for playback #ifdef KINECT_CONNECTED ofImage *img = savedImages[currentImage]; depthImage.setFromPixels(img->getPixels(), img->getWidth(), img->getHeight, OF_IMAGE_GRAYSCALE); #endif // nothing will happen here if the kinect is unplugged kinect.update(); if(kinect.isFrameNew()) // there is a new frame and we are connected { // grab the depth image from the kinect depthImage.setFromPixels(kinect.getDepthPixels(), kinect.width, kinect.height, GL_LUMINANCE); // record the depth images to "data/01/" as long as we are running #ifdef KINECT_RECORD depthImage.saveImage("01/"+ofToString(saveCounter)+".png"); saveCounter++; #endif ... } // do some cv using depthImage here ... } void testApp::exit() { // cleanup any loaded images for(int i = 0; i < savedImages.size(); i++){ ofImage *img = savedImages[i]; delete img; } savedImages[i].clear(); }Here’s how to access the skeleton positions with ofxOpenNI:
// get the first user skeleton ofxTrackedUser* tracked = user.getTrackedUser(0); if(tracked != NULL) { // get the upper left shoulder pos ofxPoint pos = tracked->left_shoulder.end }Check the header files for more info. The interface is very rough for now, so you will have to figure some things out for yourself.