The skeletal tracking that we are capable of accessing through the Kinect and OpenNI is a freaking goldmine. The initial setup of getting it all to work properly was one of the hardest parts, as there were about 8 different drivers and libraries and terminal banter that all needed to be installed before the damn thing would spit out coordinates. But once we got it all running on my laptop (but not Caitlin’s, which presented us with our second largest problem throughout the whole project) everything got going. We got coding and had a lot of examples and some of our own code up and running in no time. As a side note, I wrote up a skeletal tracking visualizer in Max/MSP, something I haven’t seen anywhere else on the web. It was a good bit of encouragement to know that I have more kung-fu than I originally expected.

The original plan was to create a series of drawn puppets that would be controlled by two or more users. Our first idea was to use Kitchen Budapest’s Animata software, but we realized that there would be no easy way to include our most important feature, multi-user interface, within Animata. It became apparent that we were, more or less, going to have to write our own Animata-like program in Processing. Starting with Sensebloom’s example Processing app “Stickmanetic” that came with OSCeleton (a basic stick-figure gui for the skeleton tracker), we tweaked it to be the controls for our puppet.

As for content, we wanted to go with something that wasn’t humanoid. Having a human puppet, i.e. waving an arm and seeing an arm wave, would be pretty banal. Thus, we decided to go with something mythological. We originally wanted to do two or three puppets, so the pair of Old Testament beasts, the Leviathan and Behemoth, seemed spot on. Behemoth was to be a two person puppet, and Leviathan a four user. We had a three user dragon in there as well, but as soon as the project’s scope would be somewhat unfeasible, we decided just to make one kick-ass Behemoth and maybe come back to the other two for a future project. The Behemoth puppet itself is a drawing I did in Flash using Caitlin’s (amazing) original puppet sketches.

After chugging along and getting the parts following the skeleton points, we came upon our single most debilitating hurdle: rotate. No matter what we tried (and try we did for hours), we couldn’t get the images to rotate around a fixed point, which is what we needed to be able to chain the puppet parts together with rivets. We spent most of the night trying to get rotate working and it just never did. Things were always showing up in completely illogical places, if they even showed up at all.

Finally we decided to go for broke and throw together a completely linear puppet (just placing the images on the skeletal points as is). The thing looked disconnected and jumpy, and the legs were attached with god-awful elastic orange bones. But at least we had something:

During the second full day of working on this thing we finally found a massively overcomplicated but functional way of computing the rotation using trigonometry. When all was said and done we had one humongous trig function that got the knees working that looked something like this:

Although the function ended up working and we eventually got a rough but pretty solid model working. Albeit choppy, everything worked like it should. 72 hours, one all-nighter, and many many expletives later, we had both a functional puppet and an earth-shattering realization: the exact solution to the problem was in the goddamned examples folder. And to just rub some salt in it, it happened to be under “basics”.

And thus we embark once again to rebuild this thing in a much better and much simpler fashion. Because, as Golan says, “when it sucks less, it will suck a LOT less”.