This is only a hello world example of ofxCV + ofxUI demo.The white lines show the motion direction, ofxUI provides the interface to control the video window. To compile the two addons together without errors, I actually downloaded two more addons, which are ofxOpenCV and ofXmlSetting.
Github:https://github.com/wfdiao/CVplusUI
The goal for the Dual Hello project was simple: get two different addons in openFrameworks to compile together. Simple and Straight forward, or so I thought. The app i ended up creating changes gravity based on the expression of your face. However, for some reason the objects do not perfectly collide…but I’ll fix that… The hardest part for this project was actually getting the addons to work by themselves. I knew once I got that down pat I would be good to go. What made getting the apps to work hard was pretty much my lack of C++ knowledge in trying to get everything pointing at the correct objects. After I got the apps to work after switching editors and installing and re-installing, I though the app should be a fun interaction with one’s body. So I used the face tracker addon’s expression tracker to save expressions and depending on which one you chose, it move the shapes every which way.0
This is a mashup of two openframeworks addons to create an augmented CV project which makes the person using the application cry every time. I combined oFxFacetracker and oFxBox2d, track the location of the eyes, and then the tears start flowing.
Like this is you cry every time. :’)
Video: Here
My interactive map ended up using two addons (and a third addon just as a dependency):
ofxCSV was necessary to parse the data, and since other students in the class recommended it and had already got it working (and were able to kindly assist in my questions and confusion), it seemed like a good option. I ended up figuring out how to:
…in the process of getting this up and running and able to process my file. So it actually served as a good general introduction to “How to C++ the openFrameworks way”.
I added ofxUI later, so I had to deal with the interesting issue of how to add an existing addon to a project. I couldn’t figure out at first how to do so, and it turns out that in addition to adding a new addon to your addons makefile, you also need to (if developing in Xcode) symbolically add the addon directory to your app’s working directory. ofxXmlSettings was just an ofxUI dependency. After figuring out how to add more addons, using both went fairly simple. ofxUI has the advantage of being very well-maintained and established, so there was no explicit web documentation but a ton of examples that were provided with the addon.
I’m kind of surprised how much of the project was really just wrangling with the addons and figuring out the functionality available to me, but it was pretty instructive. A lot of the work went into the unglamorous parts of “how do I even write this single line of code”, but the addons were fantastically helpful in making a ton of functionality much easier and more accessible to do.
Embedded above is a video of my ofxAddons combo in action. The program finds a face using ofxFaceTracker and scales the face according to the loudest frequency found using ofxFft. I adjusted a threshold according to my very limited vocal range so that when the fundamental frequency of my voice was lower than 310 Hz my face would expand, and when above that frequency, my face would shrink. There is also a volume threshold so that only loud noises picked up by the microphone scale the face detected. It was a silly exercise mainly executed by editing code in one of the ofxFaceTracker examples and adding in the ofxFft to analyze the microphone input.
Basically I did a very very simple implementation of ofxSpeech and ofxBox2D to create a ‘poetry visualizer’–when the user speaks into the computer, their words are spawned in the middle of the screen and float up to the top. The result is a muddled and hardly decipherable cluster of words from the poem bumping gently against each other. Unfortunately, the speech recognition addon doesn’t work very well (it was a miracle that I was able to record a mostly accurate run) and in order for the program to recognize the words you have to enunciate your words really awkwardly–if I were to revisit this project in the future I could try to improve it by using the Google Speech-to-Text tool which is apparently pretty accurate (though I would have to learn more about OSC).
The poem is Devotion by Robert Frost.
Github here.
I’m interested in using the Leap Motion and Kinect for multiple scale/resolution 3d tracking. I’d had some vague discussions about the trying to construct a sign language detector and am interested in seeing whether or not available 3d tracking technologies would be usable. I haven’t really done more than getting both the sensors to run, but am looking forward to doing more.
Sketches below were just for mashing together the two visualizations.
This project was very rough to get started. After sifting through an uncountable number of addons to find ones that a. I liked and found useful and b. worked, I had a few addons at my disposal. As usual, I leaned toward attempts at an audio project. I initially wanted to have some sort of interactive music player, where one would play the notes, and then have a related visual being manipulated. This was for the idea of connecting audio with visual, but I couldn’t find any visual addons that cut it. Long story/long few weeks later, I settled on ofxTonic and ofxBox2d. My burgeoning idea was that when music is made or played, the notes not only flow together, but play together. I found that when the Box2d circles drop and bounce around, they interact with each other quite a bit, like musical notes do. So, I decided to combine a Tonic keyboard with musical Box2d shapes to have the notes “play” with each other after being spit out.* This is what I produced:
Github: github.com/chanamonster/TonicBox
*Musical notes: (pun slightly intended) I initially had the keys go from A chromatically up to G#, but the horrible sounds of minor seconds made me switch. I mixed up the keys to form chords between adjacent three keys so hopefully the sound produces will at least be minor or major triads.
[youtube=https://www.youtube.com/watch?v=-dmGx-TC4EY]
For this assignment, I ended up creating a very basic 3D map to get up to speed with OpenFramework’s OpenGL and 3D meshes as this was all new to me. I originally wanted to filter the data to reveal how hotel chains were connected, as well as how flight patterns correlated with the hotel data. Unfortunately, after lots of experimentation, much of that did not pan out as I expected in the timeframe allotted.
The addons used in this project were ofxCsv and ofxGrabCam, which allows for dragging and panning of the map.
Initially, I had gotten ModestMaps to work after fixing some dependency issues, but it proved to be too slow handle the 500,000 data points in the hotel dataset. On the plus side, it was a tile-based mapping library that performed very similarly to Google Maps, which may come in handy for projects without so many data points.
I then organized the hotel data by chains using a hashmap and connected the vertices, but that caused everything to be too thick to view.
I also downloaded and parsed the OpenFlights airport and flight plan datasets and structured them in order to visualize, but also did not have time to make that happen.
Overall, even though I would consider this incomplete, I learned quite a bit about Xcode and OpenFrameworks. I seem to have gotten the hang of debugging errors that come up when integrating new addons into an existing project and I was able to help quite a few students with debugging their work.
My mom purchased a LeapMotion right when they came out and used it a bit then gave it to me since she does the standard set of tasks on the computer and there wasn’t a compelling reason for her to use it. So I have had this LeapMotion for the past 6 months without any idea of how to use, so seems like a perfect chance to actually use it!
I have seen the classic use with the now build-in ofxKinect where you use your body to play with boxes. This is sort of in a similar vein where boxes and spheres are rained down from your finger tips.
Github: Code