Some Full-Body Interactions with Cameras and Screens
Erkki Kurenniemi, DIMI-O (1971)
Used the body, as tracked by the camera, as the basis of a musical instrument.
Dimi-O (1971) is based on an optical interface, the purpose of which was to read sheet music graphically. The instrument could be played with a conventional keyboard or via a video camera. Dimi-O was also used in tandem with a dancer, whose movements were transformed into music.
Myron Krueger, Videoplace (~1974-1989)
By contrast, represented the body in a virtual environment, endowed with new powers. This is not only some of the first full-body computer-based interactive art to use a camera; it is some of the first interactive computer art, period. Krueger sought to make computing a full-body activity. Keep in mind that in 1974, the mouse had not even come into widespread use.
Two people in different rooms, each containing a projection screen and a video camera, were able to communicate through their projected images in a “shared space” on the screen.
David Rokeby, Very Nervous System (~1986-1990)
Another use of the body in a musically instrumental way, but Rokeby eliminates the screen entirely, and instead surrounds the body by a tightly responsive sound-environment.
“I created the work for many reasons, but perhaps the most pervasive reason was a simple impulse towards contrariness. The computer as a medium is strongly biased. And so my impulse while using the computer was to work solidly against these biases. Because the computer is purely logical, the language of interaction should strive to be intuitive. Because the computer removes you from your body, the body should be strongly engaged. Because the computer’s activity takes place on the tiny playing fields of integrated circuits, the encounter with the computer should take place in human-scaled physical space. Because the computer is objective and disinterested, the experience should be intimate.”
Jim Campbell, Hallucination (1993)
A mirror (less so a virtual environment) which re-presents the body undergoing a terrifying transformation.
http://www.jimcampbell.tv/portfolio/installations/interactive_hallucination/
Rafael Lozano-Hemmer, Surface Tension (1992)
Position is mapped to position. Simple and effective.
Camille Utterback & Romy Achituv, Text Rain (1999)
A classic work. The body surrounded by responsive virtual objects.
Daniel Rozin, Wooden Mirror (1999), Peg Mirror and Weave Mirror (2007)
Literal mirrors–but with sculpturally-expanded concepts of pixel-based screens.
Scott Snibbe, Boundary Functions (1998)
The Voronoi plane partitioning algorithm is used to illustrate personal space.
Brian Knep, Healing Series (2003-2009)
Where Snibbe explores the meanings made when people interact with the Voronoi algorithm, Knep explores the expressive potential of the body an an input to simulated reaction-diffusion.
Tmema, Blonk + La Barbara, Messa di Voce (2003)
Bodies (and voices) interact with simulations to produce both sound and image.
Chris O’Shea, Hand from Above (2008)
A giant hand that plays with you on the street.
Shadow Play
Scott Snibbe, Make Like a Tree (2005)
Rafael Lozano-Hemmer, Underscan (2005)
Philip Worthington, Shadow Monsters (2005)
Golan Levin, Interstitial Fragment Processor (2007)
YesYesNo, Night Lights (2010)
Influence of the Kinect
The Kinect depth sensor obliviates many of the hardest problems in vision-based body understanding. Within days of its release, it was seized upon by new-media artists eager to explore its possibilities.
Robert Hodgin, Body Dysmorphic Disorder (2010)
Karolina Sobecka & James George, Sniff (2010)
Chris Milk et al., The Treachery of Sanctuary (2012)
Design-IO, Puppet Prototype (2010)
Theo Watson is a co-founder of openFrameworks, and created the first openFrameworks interface to the Kinect, in early November 2010. Just days after the release of the Kinect, Theo and Emily Gobeille, his partner in Design-IO, created this quick prototype.
A quick installation prototype Emily and I hooked up with the libfreenect Kinect drivers and ofxKinect. The system is doing skeleton tracking on the arm and determining where the shoulder, elbow, and wrist is, using it to control the movement and posture of the giant funky bird!
This led to a commission to produce a larger work, Puppet Parade (2011)
Design-IO, Night Bright (2011)
Night Bright is an interactive installation of nocturnal discovery where children use their bodies to light up the nighttime forest and discover the creatures that inhabit it. Listening to the creatures’ sounds children can locate them in the forest, as they play a nighttime game of hide and seek.
Design-IO, Weather Worlds (2013)
Weather Worlds is an interactive installation that grants children weather controlling superpowers.
Utilizing a camera and real-time greenscreening, the installation allows children to see themselves immersed in an interactive and dynamic environment. The custom computer vision system tracks the heads, hands, feet and movement of children on the platform and also recognizes gestures. Using their bodies children can conjure a storm, release a twisting tornado or rain down bolts of lightning from their fingertips. There are mighty wind fields to move through, stomping earthquakes, light bending sunshine and blizzards that will make you shiver!
One of the core advantages of the Kinect is that it provides a skeleton for the body. This labels the parts of the body so that one knows, for example, the location of the head, the location of the arms, etc. Once you have this, it’s easy to make conceptual transformations based on such identities. Here, for example, is MoMath: Human Tree, a fractal body experience created by
the design studio, Blue Telescope:
Some Dance/Performance and Technology
Klaus Obermaier, Apparition (2004)
One of the first uses of augmented projection on a computationally-tracked body.
Chunky Move, Mortal Engine (2008)
Adrien M / Claire B, AMCB-introduction (2013) & Pixel – extraits (2014)