Simple beat tracking with openFrameworks. May be we can interact with the sound. Different beat sound represents the different order to the computer system. Maybe we can also integrated other addons such as some motion track, we can make a more complicated environment for users to interact with.
ASM face tracking addon based on Jason Saragih’s FaceTracker.
These three addons above are all about tracking and computer vision. I think they are interesting and I always think to interact with “air” maybe. Just stand their, without anything you can control the digital devices. But this will be easy if computer can see you and know your action or motion, even your expression on the face. I can use my emotion to control the digital device. This is also a very good idea I think.
This is a physical engine add-on. Box2d is a famous open source physical engine under c++. I think this can help me to make a very real interaction world and with gravity some performance will be cool and interesting.
This is an add on for open frame works that allows you to write open gl shaders (abbreviated GLSL). Shaders, as opposed to normal filters or visual effects, are nice because they are really low level, so you can do lots of fancy things all at once. A few examples of what you can do with shaders include, filters, generative stuff, and interactive fluid systems.Gonzalez did this supper cool project where he make a sand pit into a topological map using a shader. This is a link to a sandbox of running example with code. From what I can devine
With this add on plus a kinect you can turn any object into a muti touch surface. I don’t I actually want to do something with this. It is too much about making interfaces. You could orchestrate a situation where inanimate unexpected objects, such as a small garden gnome or a collection of porcelain figurines, become the controllers for an uncanny technologically mediated garden. (just a thought).
Their readme file is not informative, but from what I can gather fern is a computer vision library that sees and models images it is trained to recognize by its key features. The system can then determine how far the image is from the camera and the tilt and orientation. I’ve seen two big projects that use similar software; First there was the boards magazine project by Theo Watson and Emily Gobeille. I also saw a project that augmented peoples personal vision with glasses. I could see this software being used to identify all the media people are engaging with within a certain public space. It would also be fun to hide messages within different images.
ofxGSTT is an openFrameworks addon to use google unofficial speech to text API. It is an addon that translates speech into text for openFrameworks. There are two projects that come to mind. I think it should be interesting to use this addon in a classroom environment. Imagine a class or a lecture hall where a teacher speaks, and his speech is projected onto the wall. That speech can then be saved and given out to students at the end of the class. It will also be interesting to see what comes out of a person’s mouth over an hour or so period of time (Let’s hope that the professor doesn’t swear). Also, it would be cool to use this similar concept, but instead of a classroom, it will be used at a concert. The lyrics that you sing will be projected on a large screen, allowing people to follow along — a mix between a performance and a karaoke.
ofxMSAFluid is an addon for solving and drawing 2D fluid systems based on Navier-Stokes equations. Navier-Stokes equations describe the fluid motion, and can be used to model many things such as water flowing through a pipe and air flowing around airplane’s wings. I took a class called ‘Fluid Mechanics’ that studies fluid and the force that is being applied, Honestly, I don’t really know exactly what kind of application I would use this for, but I find it very interesting to see an application that uses the technical material that I learned in a mechanical engineering class to create work that is artistic and beautiful.
ofxMpplr is a projection mapping addon. I am pretty new to the concept of projecting virtual contents on physical objects. But after looking through videos of projection mapping projects, I want to explore more on this art form. I think this addon could make a potentially powerful work.
I think this is a well thought, and handy framework. For data visualization or for crazy CV projects,
this framework is the ultimate tool / the swiss-pen-knife. Enabling artists to stream cameras from all over the world,
and making it dead-simple to use them in an environment like OF is amazing.
First thing that came to my mind was to use this for generative music from all over the world.
Every once in a while, a new interface like Kinect, or Sifteo lands on artists’ desktops.
And it is a challenge to hack it, and make something useful out of it.
This year, Leap Motion will be released, and yet the ofxAddon is already out, even before the device itself.
Hoping to find an addon for my Emotiv EPOC EEG Headset, I checked the ofxaddons website, and found this.
However, my enthusiasm to use this addon died, right after I saw that it’s an empty github project, with just a readme file.
Challenge accepted. I will be hacking Emotiv EPOC and coding an ofxaddon throughout this semester. I think this will enable artists to use the power of their ‘brains’ in ofx.
Although not exactly a ofxAddon, ProCamToolkit is collection of apps aimed at making high level projector-camera calibration more accessible. What I like about it is the possibility to combine and calibrate multiple cameras and projectors, something I have been wanting to experiment lately. It has been developed by Kyle McDonald during Guest Research Project v.1 at YCAM Interlab.
I just can’t say enough good things about James George and his contribution to the Openframeworks community. ofxTimeline is probably one of my favourite addons, espcially when I am dealing and experimenting with generative animations and art. For example the video below is an animation I created with Openframeworks, where all the scaling, rotation, coloring has been automated using timelines.
Assignment: (Due 1/16) New Media Arts. After foraging widely (consider this list of potential starting points), select three projects in the broadly-scoped field(s) of computational new-media arts and/or design. Choose one project that you admire profoundly (why?), one project that surprised you (why?), and one project that could have been great but disappointed you(why?). (Note: all projects should somehow have involved/required the creation of custom software.)
1. Admired work
Ryoji Ikeda, The Transfinite, 2011
It is very difficult for me to name a computational new media art work that I profoundly admire. There are very many computational art work that I find technically interesting and also there are works that I admire, but they are often not of a same work. While I appreciate new technical innovation as well as creative interpretation of dated technology, the question is if the work is ‘interesting’ enough to be ‘profoundly admirable’.
“In the transfinite, Ryoji Ikeda takes the pursuits and structures of mathematics as one ‘material’ for his aesthetic, and does so with monumental and poetic result. At the center of the work is his sonic and visual re-purposing of binary code: 0 and 1. These numbers form the string codes used to represent all information in the digital world. While few of us understand just how the intricacies of this works, we are impacted by it in every conceivable way and on a daily basis.” from http://www.armoryonpark.org/downloads/Ryoji_Ikeda_House_Program.pdf
I admire early electronic compositions of Ryuichi Sakamoto because of it’s expressiveness and fearlessness toward technology. His continuing work with Alva Noto is a beautiful example of long term collaboration. I also respect recent works of Ryoji Ikeda because of the way it interprets data and programs to acoustic experience. (example) Human ears are one of the most sensitive input devices of our body and Ikeda’s work, especially when experienced in physical space of exhibition or concert hall, leaves a visceral impression. It is astonishing how such an un-human thing, such as numbers and mathematics (which can be crudely understood as numbers in relation to each other) can trigger emotions and create atmosphere. However I do not admire Ikeda’s work as I admire early works (critical vehicles and projections) of Krzysztof Wodiczko for its tactical interventions in public spaces and Critical Art Ensemble for their commitment toward investigations of network and authority. It is rare to find computational work that have theoretical propositions and critical research like that of Radical Software Group.
RSG, Kriegspiel, 2008
2. Surprise
André Gonçalves is a musician, inventor, performer and new media artist based in Lisbon, Portugal. I’ve experienced his installation work and also had chance to listen to sounds performance in few occasions. I thought his work investigates human emotions in relation to technology. It reminds of living in this age and how it feels like our smallest body movements, short moments of attentions and minuscule emotions are mediated, exaggerated and broadcasted by digital technology. The robotic installation Driven By or The Careless Self-Indulgence of Dystopia seems to prove me quiet the opposite interpretation is possible.
“The system in this installation is comprised of two entities: The first, a computer and a video camera, captures all information referring to each of the motorised vehicles: identity, position, speed, motion vectors, collision routes and other pertinent calculations. This information is updated in real time and available on request for the elements that constitute the second entity, the motorised vehicles. Based on the information received, these will be able to calculate and adjust their route, considering the position of each of the other elements. The set of instructions given to the vehicles obey to basic “Play Tag” rules, so one is chasing the others until it is successful and transfers its task to another vehicle.” from http://www.andregoncalves.info/installations/drivenby/
In this installation, complex systems of wireless communications, Toy RC Car, computer vision software (Openframeworks) and Arduino microcontroller are utilized to make a machine chasing another machine. Play Tag is a popular game for youth, but also adults seem to play similar game instinctually, while driving in busy highway, trying to get into a busy subway, or walking in the street while trying to avoid running into another person or vendor. But do machines Play Tag as well? The smart systems, automations and machine learning devices often fail to understand basic logic and create humorous and other times disastrous situations. A funny example is when an out of print book becomes impossibly expensive due to multiple retailers automatic pricing based on algorithm, such as a biology textbook ‘The Making of a Fly’ becoming 23 Million dollars.
Also security systems built with the sincerest intention can be used for the most useless data mining and becomes an abstract machine for surveillance without specific intention. Gonçalves’ work is a simple composition of toy RC cars chasing one another that are created by very complex hacking of software and hardware systems. As the title implies, we the humans are driven by the desire for techno utopia, but we end up building a world of careless self-indulgence of Dystopia instead. Thus, the complexity of the work and simplicity of it’s manifestation is an impressive surprise in this work.
3. Disappointment. Rafael Lozano-Hemmer is one of the distinguished contemporary artist whose practice is grounded on computation and new media art. His large scale installations receive appraise from critics and the public alike due to it’s spectacular aesthetics, gestures for interactivity and melancholic poetics. Lozano-Hemmer continues to produce travelling exhibitions while producing smaller pieces for gallery exhibitions and collections. While I respect his practice over all, I’d like to think about Last Breath, a new piece from 2012 with few doubts.
“Last Breath is an installation designed to store and circulate the breath of a person forever. The piece consists of a small brown paper bag which inflates and deflates automatically thanks to motorized bellows similar to those found in artificial respirators in hospitals. The apparatus hangs on a wall and is activated 10,000 times a day, the typical respiratory frequency for an adult at rest, including 158 sighs.” from http://www.lozano-hemmer.com/last_breath.php
The piece contains information of a person’s breath and recreates it bionically with motorized system. Breathing of paper bag reminds me several things such as hyperventilation, emergency oxygen mask in an airplane, vomiting and also it is a way to inhale inexpensive hallucinatory chemical such as super glue. All of the situations remind of extreme bodily condition and states of anxiety. While the artist instructs other material may be used in the place of paper bag, the real melancholy remains in the system of motorized breath and its output in terribly vulnerable material, such as paper bag. The disappointment I have with the work is not the depth of melancholy and vanitas but the limits of viewer’s experience. A person’s breath is contained within the system and viewer can only observe the mechanical recreation of the breath. It is a portrait of person’s breath and the title implies imminent death. Just like any portrait painting, the breath is frozen in time. I wish there is a way in which audience breath with them and feel closer to their presence. The project description notes the system contains simple process to capture breath, which hints there was a possibility for interactivity that artist consciously decided not to utilize. What if the output was not such an abject material, but the person’s clothes or personal item? What if audiences’ breath affects the contained breath minimally? Maybe the piece wouldn’t feel so medical and inspire to be personal, while not becoming overly explanatory or didactic.
It’s probably worth pointing out at the get-go that I have never used openframeworks, and as such I have very little knowledge about what half of these addons are even capable of. So here is my hand-wavy, blue-sky assessment of what I think looks interesting.
ofxPXCU⇣
by IntelPerceptual openFrameworks addon for Intel’s Perceptual Computing SDK (PXCUPipeline) [view on Github]
The thing that drew me to this add-on was the fact that Intel’s Perceptual Computing SDK apparently supports Nuance Language Processing. Having worked with some of the Nuance people and seen their speech to text programs in action in high-pressure situations (like electronic medical record generation), I’d love the opportunity to experiment with it in my personal projects.
Consider: a program where two people talk into a microphone on separate occasions and tell a short story of their own making (or maybe two versions of the same ‘truth’). The program merges their tales into a new, third story by looking at the sentences and substituting clauses or appropriate parts of speech. Like mad-libs, only more fluid, and actually blending full sentences into each other rather than just filling in blanks.
ofxFern⇣
by ofTheo An implementatin of the Fern tracker from EPFL CVLab [view on Github]
This is sort of cheating, since we watched the ‘magic book’ demo in class on Monday, but I fell madly in love with that project, so here I am. I feel like this addon would help me take the next step in a project I began last semester in Sequential Visual Narrative. There, I created a set of photographs of objects which told a fictional story. An evidence form was attached to each photo, which had been filled out by one of the characters in the story: a police detective. The aim of the game was to piece together what was going on in the larger narrative based solely on the artifacts the detective had amassed, and his very biased opinions about them.
When I saw what the Fern tracker was capable of doing, I immediately wondered what other secrets I could hide within those photographs. Maybe in addition to the detective’s written narratives, the Fern tracker would reveal the history of the object —hidden poems from the owner, memories, daydreams, promises… Maybe tilting the photograph would reveal a fingerprint, or a monogram … Maybe tilting the camera in different directions would reveal the opinion of a different character…
My thoughts on this addon are similar to the previous two, but harnessing Tesseract could give me the ability to interact with printed text, and once I have the text, I could do any number of language processing activities with it, illustrate it, re-format it, translate it, or just use it as a data set.
Is anything less sexy than a mySQL database? Probably not. However, I’ve got tons of the damn things though storing data from 20-30 websites I’ve developed in the last few years. They’re sort of a goldmine for things like text, image URLs, and other resources. I’m not totally sure what I could do by stripping them out into OF, but certainly large data visualizations of pervious work is one very enticing possibility.
Coming from scripting/flash world, the idea of accurate timers is extremely enticing. A problem I’ve had with both flash and js based timing solutions is their utter craptacularity due to their rather high-level implementation. Timing in processing is also pretty impossible because all changes happen through void draw() meaning you’re limited to graphical updates to fire timed events. This means you’ve got very little fine grained control over (in particular) audio events. Looking forward to playing with this one.
I took an HCI class last semester (Software Structures for User Interfaces) and I came to love state machines as a very minimal MVC-style abstraction. Basically, they let you separate views from logic really easily and tend to simplify implementation of finite state control systems. They’re especially helpful for implementing custom UI elements.
Receipt Racer is an OpenFrameworks game played with a PlayStation controller, a projector, and a receipt printer. The course is printed out ahead of your “car” and you must move your light based protagonist to avoid the tangible printed obstacles.
I found the blending of tangible and ephemeral entities quite nice. The fact that the entire history of game play persists as a pile of thermal paper at the feet of the player is also quite striking. Playing the game likely seems more serious and important as one copes with vague eco-guilt as they play.
There is room for improvement in the very typical controller. Some kind of slider or single-axis joystick would be less distracting to me. It wouldn’t tke much to bring this from tech demo to clean and sturdy installation, but it needs a cabinet.
This Graffiti analysis sculpture is presumably a 3d printed version of an interpretation of the movements of a particular instance of graffiti. A laptop running OpenFrameworks-based program tracks the movement of the artist using an LED attached to the inking device. The computer presumably interprets luminous intensity as depth and exports a 3d model. The model is refined in blender and printed in white. It is then put in a gallery to be admired.
I found the form striking. Very interesting source for 3d data. The final piece more than vaguely resembles the aesthetics of graffiti.
Audience is a really humanist piece involving many mirrors and many servos.
All of the several dozen mirrors will orient themselves in such a way to show the audience several copies of their own reflection. They follow the audience using computer vision.
I love the idea of being unable to escape yourself. The mildly antagonistic, yet kind actions of the robots really interact well with the audience.
The work of rAndom International is quite good and worth looking at if you are not familiar.
Discover and discuss three projects that were made in openFrameworks (OF). As a suggestion: For diversity, choose one project which seems like an important contribution to the field, and one project that seems like a quick but interesting sketch or experiment.
Making of “Nikola Tesla in Sound and Light” – Marco Tempest
This is a project with projection mapping and story telling on TED talk. They are using openFrameworks to make face recognition and control LED projections. It is a high combination of traditional art and new media art.
This project attempts to free ASCII Art from the confines of the screen and enable it to exist in physical space – with simply light and paint. They use real paint to convert the painted to ASCII. As the user paints with white paint on black background, software converts the painted into projected ASCII code in realtime thus creating a sensation as if the code is painted directly on the surface.
Public Souvenir
This is a simple but interesting public project with 3D printer and openFrameworks. When human profile is recorded by openFrameworks, the 3D printer builds a small miniature for models.
