Stillness (Assignment 7/9)
Overview
I made a projection of virtual butterflies which will come land on you (well, your projected silhouette) if you hold still, and will fly away if you move.
Inspiration
This semester, a friend of mine successfully lobbied for the creation of a “Mindfulness Room” to be created in one of the dorms on campus. The room is meant to be a place where students go to relax, meditate, and, as the name implies, be more mindful.
For my final project, I wanted to create something that was for a particular place, and so I chose the Mindfulness Room. Having tried to meditate in the past, I know it can be very challenging to clear your mind and sit entirely still for very long. So, the core of this project was to make something that would make you want to be still (and that would also fit in with the overall look and feel of the room.)
Technical Aspects
Some of the technical hurdles in this project:
- Capturing a silhouette from a Kinect cam image. I tried to DIY this initially, which didn’t go well. Instead, I ended up finding this tutorial about integrating a Kinect and PBox2D. I fixed the tutorial code so that it would run in the most recent version of Processing and with the most recent version of the SimpleOpenNI library.
- Integrating assorted libraries: SimpleOpenNI, blobDetection, PBox2D, ToxicLibs, standard Java libraries. I almost certainly didn’t actually need to use all of them, but figured that out too late.
- Dealing with janky parts of those libraries (e.g., jitteriness in the blobDetection library, fussiness of SimpleOpenNI). Using the libraries made my project possible, but I also couldn’t fix some things about them. I did, however, manage to improve blob detection from the Kinect cam image by filtering out all non-blue pixels (the Kinect highlights a User in blue).
- Trying to simulate butterflies flying—with physics. Trying to simulate a whimsical flight path using forces in PBox2D had only ok results. I think it would be easier to create their paths in vanilla Processing or with another library, (though that might make collision detection far more challenging.)
- Finding a computationally cheap way to do motion tracking. When I tried simple motion tracking, my program ate all my computer’s memory and still didn’t run. I ended up taking the Kinect/SimpleOpenNI provided “Center of Mass” and using that to track motion, which worked pretty well for my purposes.
Critical Reflection
As I worked on this project, I was unsure throughout that all the pieces (butterflies, kinect, etc.) would come together and/or work well. I think they came together fairly well in the end. Even though the project right now doesn’t live up to what I imagined in my head at the beginning, it still does what I essentially wanted it to do—making you want to stay still.
When people saw the project, their general response was “that’s really cool”, which was rewarding. Also, the person in charge of the Mindfulness room liked it enough that she wanted me to figure out how to make it work there long term. (Which could be really logistically difficult, in terms of setup and security because the room is always open and unsupervised, and drilling into the walls to mount things isn’t allowed.)
So, though there’s a list of things I think should be better about this project (see below), I think I managed to my concept simplistically, and well given that simplicity.
Things that could be better about this:
- Butterflies’ visual appeal. Ideally, the wings would be hinged-together PBox2D objects. And antennae/other details would add a lot.
- Butterflies movement. Could be more butterfly-like.
- Attraction to person should probably be more gradual/a few butterflies at a time.
- Code cleanliness: not good.
- Ragged edge of person’s silhouette should be smooth.
- Better capture of user. Sometimes the Kinect refuses to recognize a person as a User, or stops tracking it. This could have to do with how I treat the cam image, or placement, or lighting, or just be part of how I was doing Kinect/SimpleOpenNI. After talking with Golan, I think ditching OpenNI altogether and doing thresholding on the depth image would work best.