Overview

This project is an open source and affordable hardware platform for advanced cinematics.  These technologies for camera movement are typically only available to financially able production companies, but this device will give the same level of dynamics to anyone with a laptop.

What’s Out There

There are a large number of commercial products and diy tutorials that cover this subject.  However, there is quite a disparity between the diy/ open source builds versus the commercial.  On top of this fact, the commercial does not reach the ability to track subjects in real time until the budgetary stratosphere is reached.  This space remains entirely unexplored as an affordable means to obtain high quality, and dynamic footage.

Hardware

+What is needed?

-Stepper Motors (sound control // smoothness)

-Aluminium rail (light // strong)

-Acrylic parts (easy to fabricate // inexpensive)

-Arduino + Drivers

-Kinect

 

 

Code Schematic

 

What’s Next

+Develop Pan Tilt Mechanism

+Devise Tracking Algorithms

+Machine Connectors for a Sturdy Result

+Define Choreography

 

Schedule

March 31st– Completed Pan Tilt Mechanism [Design and Build]

-Strategy for Choreography

 April 14th– Kinect / Development Completed

April 28th– Completed/Refined Prototype for Critique

 

 

Mano-Extendida: Free-hand gestures for sound manipulation using leap motion controller

Current Status: I started this project on a linux system and have attempted to transfer over to osx with extremely poor results. First copying from git and simply running the openframeworks project resulted in errors I couldn’t work around, so I rewrote the of program to get the leap data and send it over OSC. Ok, now that that is working my pd patch should work exactly the same because the messages are exactly the same before… NOPE! For some reason, Pd-extended is only receiving OSC about 4 seconds AFTER I REMOVE MY HANDS from the leap’s view. It just gets a shitload of data spit out together. So I check and see that pd vanilla 0.46 has a new built-in osc parsing object and test it out. After the initial test it seems that this will work and proceed to completely rebuild the pd patch because, naturally, I can’t just open my pd-extended patches in pd vanilla and I can’t copy/paste either, great. After that was done with, what do you know, Pd is totally shutting down my OSC parsing, this time when DSP is turned on! The same thing as before, about 4 seconds after removing my hand from the leap’s view, I get a lumped together a bunch of osc data. I tried installing jack to see if the issue is with pulseaudio but the result is the same. That’s where I am now, totally frustrated.

On a different note, I’ve been giving some thought to the gestures I want to implement for sound design and manipulation. Here are some illustrations, hopefully they’re self explanatory and thus appropriate for the task.

Here’s one that’s not self-explanatory, but I’m inclined toward it anyway.

 

Here’s some of the research I’ve done, mainly comparing systems the use gloves vs no gloves for sound synthesis. It’s very light on the gesture side of things but I’m working on that.

For my capstone project proposal, I am performing a textual analysis of Dilbert, a comic strip about an engineer in a bureaucratic corporate machine. Since the late 80s, the author has been creating over 10,000 strips for nearly 26 years. In an earlier assignment, I scraped the dialogue of each of these individual strips and downloaded each strip as separate images; this data was used to produce a visualization indicate the relationship strength of each character pair. For my capstone project, I would like to extend this exercise to extract more interesting information from this data set and apply it to a creative application.

I currently have dialogue for each strip, but the dialogue is not associated by individual panel (there are three panels on a typical strip). Using Python image manipulation and optical character recognition (Tesseract) bindings, I intended to slice each of the strips into individual panels and then perform OCR to associate dialogue with a specific panel. Levenshtein Distance algorithm would be used to determine the OCR text with the ground truth of the scraped dialogue. Once this task has been completed, I can wipe the original dialogue from the strips’ image files. Then, using natural language processing techniques, I can compare the topic/subjects of an individual strip with other bureaucratic content, such as CSPAN transcripts or United Nations assembly meeting minutes, and then insert this new text content with a Dilbert-like font to effectively create a new strip.

Part of my previous research looked into what other information I could extract from the comic strip’s dialogue. They include my Looking Outwards 9 post, which showed examples of plotting the character appearance frequency in a given chapter of Les Miserables and noting the setting’s mood in which the character appeared. Another example in the Looking Outwards 9 post showed a visualization of common phrases or words spoken by various pubic speakers. At Golan’s suggestion, I have also looked into MALLET (MAchine Learning for LanguagE Toolkit), which provides a way to analyze large amounts of unlabeled text and group them into topics, which are determined by analyzing cluster of words that frequently occur together. This toolkit can be applied to bring out common themes and topics that occur throughout the Dilbert strip and use this data to determine what type of content can replace the strip’s original dialogue in a way that makes sense.

A 140-character description of this project can be worded:

My proposal is to textually examine dialogue from every Dilbert comic strip and replace them with new content from outside sources.