This project uses wood pieces to create a mirror that interacts with people as they walk in front of it. A total of 830 wood pieces and motors are used that rotate vertically, changing their color as the user walks in front of the mirror.

I found the project interesting because of its novel use of wood, a non-reflective material, as a material for simulating a mirror. I am also surprised at how realtime the mirror is, despite its use of mechanical motors for each wood piece. It is also very creative in using the variation in brightness when a wood piece is rotated under light.

I think the project would be really cool if lenticular pieces were used instead of wooden blocks. That way, the blocks would actually change color (instead of brightness) when they are rotated.

Laika Font – interactive kinectic type

I love how this piece explores new ways of creating type, even though it is limited in its mapping of movements in how it translates gestures to font characteristics. It’s a perfect next step in terms of typography. The first faces of moveable type were based on the movements of hand written script. It is only very recently that grotesque, humanist, geometric fonts have becoming conventionally acceptable, as their lack of relationship to the human hand were originally found unpleasant. The concept of allowing human, physical interaction in a different way to inform the creation of font characteristic could be quite inventive and beautiful. It could meld strong computation with physical markmaking of the human body — what if you could dance a typeface, or a yoga position, or your pet ferret.

DaVinci by Razorfish

This game allows people to draw shapes which the program then understands and renders as weighted physical objects. Unlike funky forest, which provides a highly stylized environment, DaVinci creates shapes that are fairly direct translations of the participants gestures. As a paradigm for interaction, it is really lovely, and suggestive of uses — creating games where participants have to move through obstacle courses, or drawing elaborate Rube Goldberg machines. This environment allows the participant to create highly functional (though perhaps not useful) creations that act, with little overhead.

Funky Forest by Theo Watson

I learned about this project at the openFrameworks lectures earlier this semester, and responded to it because I love how it creates an immersive experience. The installation allows viewers to create and maintain a forest, where they can grow trees with their bodies, interact with flying animals on the walls, and move water on the ground. I appreciate the integration of using real, physical objects to interact with the simulated worlds as well as physical gestures which nudge people to interact in ways that the normally do not. Like my previous post,  this project allows the viewer to participate in the creation process. There is a careful balance of creating constructed rules for the environment and interaction with creating an open ended environment that allows the viewers to interact as they feel.

1. Philip Worthington – Shadow Monsters, Responsive Environment 2005

This is definitely an impressive project. Creating shadow puppets is a great concept. I sometimes have the thought that if I have the ability to control the shape of my shadow that will be awesome. I am glad this project made it partially come true. Maybe using shadow to create a fighting game is a great idea?

2. Daniel Rozin – Mirrors Mirror, Motoristic Responsive Sculpture 2008

The idea that using tiny mirrors as pixels is smart. The property of mirror made this project really fun. If using kinect to track a full body movement could be more interesting? Or, the installation could beyond 2D surface? Like distorting mirror?

3. Kinect Rubber Sheet

One of the best things of kinect is that it allows you to mix the virtual world with the real world. This project is an example. By pushing your hands towards kinect, from the screen, it seems that you are trying to tear a rubber sheet of yourself. I am obsessed by these mix reality things.

6: Word Lens

Word Lens is an augmented reality application designed to translate text in a foreign language to one that the viewer can understand by overlaying the translated text on the mobile device screen where the original text should be.

This project is nifty because it attempts to create a more authentic experience for the viewer by preserving the context of the words instead of spewing it out as simple text. It also attempts to translate any text in the camera’s field of vision on the fly, surpassing many other applications that require a still image to be processed. They adhere to the text color pretty well, but it would be nice to improve the preservation of type style.

[ Word Lens at Augmented Planet ]

7: BendDesk

BendDesk combines two projectors and three cameras to make a superior workspace, allowing you to handle digital documents similarly to physical documents. The project is awesome because it removes the limitations of previous implementations done only with common interface devices (e.g.: virtual desktops with stacks of documents that you interact with via a mouse and keyboard). The missing component was always the freedom to interact with the document using your body as the manipulator, and now users are able to drag the document around with their hands across two planes. It would be great to incorporate another camera that could project the user or another user on the vertical plane for a cool teleconferencing environment.

[BendDesk]

8: Teleoperation of Humanoid Robot

This project has a person controlling a humanoid robot through a kinect. I think that the implications of the project are cooler than this base implementation. For example a person could exhibit fine motor control in a remote location, eliminating the need to actually travel to the location. This has excellent potential for something as important as specialized surgeons operating on patients in remote areas or a person attending a conference able to more actively participate in subtlety important human transactions like conversations in a hallway. While a nice basic implementation, the next steps would be to increase the size of the robot and improve the accuracy of its corresponding motion.

I am interested in some existential uses for kinect that do not have a set game logic. I am envisioning an infinite swimming virtual space.

Building and abstracting virtual spaces that I could convert into game maps and then play them. More for 3d output/simulation assignment.

Background subtraction + kinect = greenscreen bathroom. Want this >>

Andy Wilson is a senior researcher at Microsoft Research. He was doing Kinect-like projects before the Kinect came out. This one in particular involves a digital rc car being controlled by xbox wireless controller driving around in digital world. That digital terrain you are navigating can be altered by placing physical objects on the projection surface. The physics of the car changes as ramps, blocks, and mounds are introduced. This is a great example of table top digital gaming that will be the future.

Much of his new research about depth sensing as touch sensor is very interesting

See his site here with more videos

Dot matrix display is cool. I like that how this is so high tech and yet so low tech at the same time. There’s something attractive about the abstraction of the form on the wall of OLED panels.

Posted this for no other reason than the fact that it is Street Fighter and kinect. On a serious note, it is interesting how physical motions maps very naturally to games that uses physical motion (duh) but I wonder if the kinect can be responsive enough to make playing a game like this even mildly competitive. However, this comes to a very obvious shortcoming of the kinect: when used in a setup like this, the user can only move within perhaps 3 feet by 3 feet square, which means that Kinect games might be a largely stationary affair.

Cheese is a set of recordings of actresses trying to hold a smile. A computer continuously “rates” the strength of their smile, and sounds an alarm if they falter. It’s technically impressive. I’m used to computer-detection of human expressions being jittery and fickle, but the ratings plunge immediately when the actresses begin to relax. We tend to consider smiles idiosyncratic and personal, an attitude which is threatened by objective measurement.

(Skip to ~50 seconds in.) This is a pretty minor tech demo of putting a person and a virtual character into the same space. It turns the Kinect’s field of view into a middle ground between reality and virtuality. The proximity between imagined and actual is exciting.

Mehmet Akten’s Webcam Piano is a beautiful both visually and acoustically, but it unfortunately is not nearly as interesting as a real piano, because it gives the user no control. The notes are all pre-set to a pleasing scale so that there is no possibility for dissonance, and the compact arrangement notes prevents granular control. Instruments and art-making devices should ease, enable, and respond to the choices of the user, and not try to make up for the user’s lack of talent.

A very simple presentation using kinect and openframeworks. Not very interesting style wise I think, but there’s a few things to note here. Firstly, it is kinda similar to classic CV projects such as text rain and even the works by Theodore Watson but using the Kinect now, it means that these kind of installations can now potentially work for a much larger audience. As seen on the projection, the shapes of individual humans can be made out and therefore interaction can be separated. I think that’s something interesting and potentially can be exploited in the future.

Unknown Kinect Pleasures v2

A minimalist Kinect visualization demo by Gregg Wygonik. In the midst of its simplicity, this demo manages to exhibit a range of visual variation through fairly small modifications to the code.

Kinect VR

A virtual reality demo using Kinect in which spatial depth is used to determine the view relative to the angle you are positioned at.

Kinect Piano

I don’t know how practical this is, most would probably prefer the tactile feedback associated with playing a real piano, but its an interesting demo that suggests some applications for hand/gesture recognition with virtual or augmented interfaces.

I am in love with this project, the virtual marionette, which uses the Kinect to do real-time puppeteering of a humanoid marionette; it attempts to stay true to traditional marionettes, while bringing tradition into the technological age. I would love to work on something similar, riffing off of finger puppets or shadow hand puppets; I’m an animator at heart, and I find real-time animations that mimic the user very endearing and compelling.  I’ve seen similar projects by Theo Watson like this funky bird puppet, but the amount of articulation in the marionette project is particularly wonderful.

More about the Virtual Marionette

Audience – rAndom International from Chris O'Shea on Vimeo.

Audience is an amazing project that uses vision tracking to then mess with the users vision. Its quite clever in that once you approach the mirrors orient on a normal from your head and thus you see yourself. The conceptual play of watching and be watched makes for an experience that is unique. I very much appreciate the cleanliness of the hardware aspect and the swarming aspects of the mirrors. The software used was openFrameworks. A beautiful, simple project.

find out more here… audience

Nevermind the safety concerns (the police did). There is something absolutely exciting about exploring a city (especially one with height) through a lens that offers us a perspective unlike any other. This project becomes very interesting when you begin to think about what google earth has done for digital-urban exploration. Techniques like these offer us new ways to think about presence, urban exploration, delivery, location. Unlike google street view you are witnessing a real-time camera feed and can react with the real-time environment. I like the fact you wear the googles and fly a plane, reminds me of my youth…ahh Microsoft Flight Simulator 98

Pilot claims in interview he has 120 mile range with control and camera feed. Awesome.

interview with pilot here

Keyboard Anywhere

This is a project that allows you to place a virtual keyboard onto any surface, such as a desk or the floor. It can be played by pressing or stepping on the surface. I think it’s a really interesting idea because it is so flexible in terms of size and location. You can create a huge keyboard for fun or a small keyboard for actually (pretending to) play the piano. A big issue with this project is that you need some sort of visual cue in the real world to know what keys you’re pressing and where the other keys are located, but this could be fixed with a projector or even just some paper taped to the ground.

So I’ve been thinking a lot about what I want to do with the Kinect. I got one for christmas, and I still haven’t had time to do much with it. I’m a huge music person, and I’d like to create an interactive audio visualizer that takes input from body movement instead of perceivable audio qualities (volume, frequency waveforms, etc…). I think that using gestural input from a person dancing, conducting, or otherwise rocking out to music would provide a much more natural input, since it would accurately reflect the individual’s response to the audio. I can imagine pointing a Kinect at a club full of dancing people and using their movement to drive a wall-sized visualization. It’d be a beautifully human representation of the music.

I’ve been Googling to see if anyone is doing something like this already, and I haven’t been able to find anything really compelling. People have wired the Kinect through TUIO to drive fluid systems and particle emitters, but not for the specific purpose of representing a piece of music. I don’t find these very impressive, because they’re really dumbing down the rich input from the Kinect. They just treat the users’ hands as blobs, find their centers, and use those as multitouch points. It must be possible to do something more than that. But I haven’t tried yet, and I want everything to be real-time – so maybe not ;-)

Here are a few visual styles I’ve been thinking of trying to reproduce. The first is a bleeding long-exposure effect that was popularized by the iPod commercials a few years ago. Though it seems most people are doing this in After Effects, I think I can do it in OpenGL or maybe Processing:

This is possibly the coolest visualization I’ve seen in a while. However, it was done in 3D Studio Max with the Krakatoa plugin, and everything was painstakingly hand-scripted into a particle system. I love the way the light shoots through the particles (check out 0:16), though. I’d like to create something where the user’s hands are light sources… It’d be incredibly slick.

I’m not sure how to approach implementing something like this, and I’m still looking for existing platforms that can give me a leg-up. I have significant OpenGL experience and I’ve done fluid dynamics using Jos Stam’s Navier-Stokes equation solver, so I could fuse that to a custom renderer to get this done, but I’d like to focus on the art and input and let something else handle the graphics, so suggestions are welcome!

Very psyched to start messing around with the Kinect! It’s amazing that such a huge amount of things can be done with a damned console peripheral.

Looking through the MediaArtTube playlist was really interesting. Daniel Rozin’s motorized mirrors are really amazing devices, and the wooden one is an easy favorite. The material is wonderfully natural in a Scrabble-tile kind of way, and the wood makes for a really interesting sound texture for the sea of clicking coming from the rig.

Our nerd compatriots at MIT fashioned this Minority Report style interface with the Kinect. This is a really awesome technology that I can see becoming an actual reality with some serious time and effort.

Another awesome Kinect interface in a similar style is the physics based drawing application DaVinci. The user makes gestures to draw shapes on the screen as well as to perform a variety of physics-based actions with the drawn objects.

Primer For my own project, I am interested in the idea of augmenting a live feed of yourself into a different you. To start, I wanted to search for existing projects which already do this.
More on this later in the napkin sketch post, but I am fond of this idea because it is a prevalent experience in games and because there is obvious appeal to escape through an alternative ego. (How people did you see make themselves a Scott Pilgrim or Madmen avatar?)

Update From class today, it seems like the avatar notion is well explored. Shifting focuses…

Also, this is just a photograph series, but has some applicable potential:

—

1 | Embody an Avatar (Second Life)

Exactly what it sounds like: this project uses the Kinect in order to take movements in from people to control a 3d second life avatar. The avatar mimics what the user does. Additionally, some gestures also work as camera controls.
The movements are still limited as the motions are mostly done with the upper-body in this demo. Jumping is not actually done by jumping, but by raising the hands up as if you were jumping. Rigging up a 3d model isn’t easy and probably is a bit out of scope for a 2 week project. However, the idea of controlling an alternative self has obvious appeal here.

2 | Body Painter

Body Painter — Kinect in Quartz Composer from resprung on Vimeo.

Paint with rainbow, drippy brush on a canvas via body gestures. The brush maps to one of the users hands and follows it around.
The kick for paintbomb is a nice bonus. This adds on nicely to the idea of “alternative universe” because it empowers the user to control the brush.
Most of the body is not involved in this project (only one hand and the feet when kicking). The brush options are also limited to that one brush. The critique ultimately being, this project could have so much more capabilities.

3 | Imaginary Animals

imaginary animals from phil archer on Vimeo.

A non-Kinect project. This installation leverages the mobile phone’s sensitivity to infrared light (which we cannot see with the naked eye) to reveal medieval beasts.
This is inspiring because it is a (relatively) simple hack using a device most have now and it’s done well. The discovery process is very nice, but I find that the display is too static. If the display was more animated or had another level of interaction, I think it would make the imaginary animals more alluring. For example: Maybe hiding the animals in a larger canvas and then giving the user the ability to interact with them as a reward.