For this project, I created an app which takes a person’s face, and generates a sine wave based on a few characteristics. Here’s a picture of me testing it out. There’s no way to look graceful with this thing:

Here’s a video of our esteemed classmates trying it out:

My biggest takeaway from this project was that sometimes it’s easier to work things out on your own, rather than fumbling around with an addon that promises it will work. I had been working with ofxTonic, but memory issues in the addon download made my app crash soon after it was launched. I tried other libraries, but none worked well enough for my purposes.

Thankfully, we have some electrical engineering classmates, who helped me tremendously by teaching me about the math of sine waves. With their help, we populated the sound buffer manually, rather than trying to get ofxTonic up to speed. It works really well!

(Sketches soon, when my phone sends the email)

GitHub

Description this assignment has been much more frustrating than what it could have been. Installing visual studio on windows and running the project generator were the biggest challenge. Another challenge was finding openFrameworks addons compatible with visual studio and free of bugs. After several trials and failures, I decided to switch to Mac and run xcode. At the end of a long day of testing ofxAddons and trying to find creative ways to put two together, I had a dream about a project idea where OpenCv and Box2d would work together to create an interactive software where shapes would rotate around real-time moving objects. And I turned my dream into reality…

Github

In this project, I thought it would prove to be a good idea to use it as a dry test run/proof of concept towards my final project idea. Right now, for my final project I want to create a fully interactive stage. This will be centered around a pepper’s ghost stage, which will help augment reality even further creating a hologram-like experience for the viewer. The two Addons I used for this project where ofxLeapMotion and ofxSyphon.

ofxLeapmotion was a fun addon to play around with. It is also very powerful and full of potential. The one thing I did find out though is that it is very sensitive. Sometimes it thinks it sees a hand when there shouldn’t be one in its view, and sometimes it just creates a ghost for no reason. BUT besides all of that, It proved to very fun and powerful once I learned how to use it more in a performance like test.

The second addon, Syphon, was a must-learn for me. This is due to our projection mapping software of choice in the Media Department relies heavily on Syphon inputs. I figured if I was going to code anything for the theatre, it would eventually need to be able to output to other programs in an efficient way. So I thought this was the best time to go ahead an learn it!

This project consisted of me using the LEAP to control a custom 3D object (I made a 3D model of this year’s School of Drama Centennial 100 Logo) in 3D space while outputting through Syphon to be projection mapped using Millumin (our mapping software of choice in the department). This was then sent out to be projected onto the ground, to be caught into the reflection of the Pepper’s Ghost stage to create a hologram like object floating around my body. In the end, I decided to make a fun tango for the occasion using the 3D object as my dancing partner! I had a lot of fun (maybe a little to much… nah, never to much) with this project and learned a ton while doing it!

LINK to my Github Rep:

https://github.com/kdloney/LEAP_PeppersGhost

I had such strange problems getting started with OpenFrameworks that by the time I got to trying to make the add-ons work I just didn’t have time to deal with the lack of Windows instructions, so I did this assignment with Processing libraries instead. I chose two libraries that I found interesting but probably wouldn’t actually use together- blob detection and cell noise. I think cell noise is really interesting as a natural phenomenon and can make some really beautiful things. Blob detection seems like it could be useful for lots of interactive projects.

The biggest thing I learned from this project was that if I intend to use OF add-ons I should decide to get help or use the macs in the studio really early in the project…

Here’s a  video of some more generated noise.

Thereface is a computer base theremin which is controlled by the position of your face. It works by using ofxFaceTracker to find a face on the screen, and return the position in pixels of the face. The top left corner being (0, 0) and the bottom right corner being (640, 480), which is the application’s screen size. The face position is then used to compute both a frequency and a volume to be used as a pitch. The pitch varies from 200 Hz from the far left to 1000 Hz at the far right. The volume ranges from 0% at the top to 100% at the bottom.

The audio is synthesized using ofxTonic, which is an openFrameworks port of Tonic Audio. Tonic Audio is a sound synthesizing library for C++, which allows easy synthesis of various types of waves. Thereface uses a simple sine wave. We update the sine wave’s frequency whenever we detect the face as being in a new position.

Please note that in this video, the audio sometimes stands still while my face is still moving. This is because the face tracking software stopped recognizing my face. I blame the beard. Also, I apologize for the application crash. Thereface worked great until I started using a screen capture software and that was best take I could get.

Github

Face Recording

As a warm-up for openframeworks I created a simple face capture application. It runs on start-up and watches my face until it is centered on the screen. Once in place a photo is taken. These daily photos will become part of the quantified self assignment. I utilized openCV and the UI addons.

The code is on Github

Sketch

I came about this idea by observing how people use their cell phone’s loud speaker and mouth shape so create Wah-Wah effects- I thought why not make it happen through OpenFrameworks. I chose ofxFaceTracker and ofxMIDI as these are well supported plugins and have Windows support as well (generally, OF Addons run on all platforms but these two have been tested on Windows as well).

It’s pretty simple, I used the FaceOSC example and added ofxMIDI to it to export face parameters to a MIDI port. I created a virtual MIDI port on my system and sent the Mouth Height data mapped as Pitch Bend- this Pitch Bend is used to vary the frequency of the low pass filter in my DAW i.e. Ableton Live.

Github Repository: https://github.com/harisusmani/Project3_FaceSynth

1. ofxUI: A User Interface Addon for OF

This is an extensive addon that allows the creation of a variation of GUIs and would be a powerful assets for creating data visualization designs.

2. ofxMSAInterpolator

I think ofxMSAInterpolator is a powerful data feed system that will enable a lot of possibilities to creative generative artwork based on real-time simulations.

3. ofxMSAShape3D

This addon enables the creation of 3D meshes on embedded systems/iPhone etc.