Since my initial proposal, I have made a lot of progress on my project. The actual project goal has not changed much, however I have scaled back some of the technical components to be more realistic given the time frame.

A quick overview of what I’m doing: I’m making an animatronic avatar for skype conversations. Use video feed from your skype conversation and computer vision to make your friend control a robotic avatar. Skype bot?

Initially I had hoped to control the eye position, eyebrows and, mouth width and height. I learned that the eyetracking required to map eye movements would be very difficult and require specific conditions (lighting) in the room to do well. Therefore I have scaled the movements down to the eyebrows and mouth. To increase emotiveness I have also added ears that are controlled by the servos in the eyebrows. This gives me more motion and expressive quality without the cost of powering and controlling more servos.

So far I have mostly completed the underlying mechanisms and skeleton for both robots. This is what they currently look like. They will eventually be covered in a more friendly material to hide the messy circuits inside.

Here’s some sketches of what they might look like with the friendly covering. I purposely made them nonhuman to avoid any uncanny valley effects which some people find off-putting.

Right now I’m working on getting the hardware and software to work. Here’s a video of me testing the example code on my servo shield. (the thresholds were not set up for my mechanisms :/) (sorry its in portrait not landscape)

My plan on the software side involves gluing a lot of different applications together. It’ll go something like this:

SKYPE >> PIXEL GRABBER SOFTWARE >> OF FACEOSC >> OSC >> ARDUINO >> SERVOS

I’ve found a 3rd party pixel grabber software that successfully sends video to OF.

To help with debugging in OSC I’ve created two OF apps that simulate the Facetracker output (OSC sender) using GUI sliders and Arduino response (OSC receiver) by moving a little face I drew. So far everything works well.

Now I’m working on sending OSC messages to an Arduino from OSC using OSCUINO (thanks David Newbury!) and the OSC library on Arudino. So far I’ve successfully run example code written by David and am moving to scaling up the Arduino output to match the outputs on the face.

Once that’s working I can move onto swapping out the slider output for real FaceOSC output.