#5 (Body, Due 9/25)

#5 (Due 9/25)

This set of Deliverables (#5) is due at the beginning of class on Wednesday, 9/25.

  1. Optional Reading: 2019: A Face Odyssey (no response required)
  2. The Augmented Body

2. The Augmented Body

For this project, you are asked to write software which creatively interprets (or responds to) the actions of the body, as observed by a camera.

You are asked to develop a computational treatment for motion-capture data captured from a person's body. You may create an INTERACTION (by using real-time motion capture information from a body-tracker or face-tracker), or you may create an ANIMATION (by motion capture data calculated from a pre-recorded video).

Template code is provided for both scenarios, in the lecture notes for 9/18.

Some Creative Opportunities

There are many possible ways to interpret this assignment. The following suggestions, which are by no means comprehensive, are intended to prompt you to appreciate the breadth of the conceptual space you may explore. In all cases, be prepared to justify your decisions.

  1. You could develop a (software) mask or costume, and demonstrate it through a deliberately composed performance.
  2. You may work in real-time (interactive), or off-line (animation). You might choose to develop a piece of interactive real-time software (as in Setsuyakurotaki, by Zach Lieberman + Rhizomatiks, shown below). Or you could choose to develop a piece of custom animation software, which creatively interprets skeletons calculated from a pre-existing video file.
  3. You may use more than one body. Your software doesn't have to be limited to just one body. For example, it could visualize the relationship (or create a relationship) between two or more bodies (as in Scott Snibbe's Boundary Functions or this sketch by Zach Lieberman). It could visualize or respond to a duet, trio or even a crowd of people.
  4. You may focus on just part of the body. Your software doesn't need to respond to the entire body; it could focus on interpreting the movements of a single part of the body (as in Theo Watson & Emily Gobeille's prototype for Puppet Parade, which responds to a single arm). Some of the provided templates (PoseNet) provide information about the full body, while others (clmTracker) provide information about the face.
  5. You may focus on how an environment is affected by a body. Your software doesn't have to re-skin or visualize the body. Instead, you can develop an environment that is affected by the movements of the body (as in Theo & Emily's Weather Worlds).
  6. You may control the behavior of something non-human. Just because your data was captured from a human, doesn't mean you must control a human. Consider using your data to puppeteer an animal, monster, plant, or even a non-living object (as in this research on "animating non-humanoid characters with human motion data" from Disney Research).
  7. You may make software which is analytic or expressive. You are asked to make a piece of software which interprets the actions of the human body. While some of your peers may choose to develop a character animation or interactive software mirror, you might instead elect to create "information visualization" software that presents an ergonometric analysis of the body's joints over time. Your software could present comparisons different people making similar movements, or could track the accelerations of movements by a violinist. You could create quantitative comparisons of people's different gaits.

Summary of Deliverables

Here is the checklist of expected components for this assignment.

  • Sketch first! Draw some ideas. Study people moving.
  • Develop a program that creatively interprets, or responds to, a human body or face.
  • Create a blog post on this site to document your project and contain the media below.
  • Title your blog post, nickname-Body, and give your blog post the WordPress Category, 05-Body.
  • Write a narrative of 150-200 words describing your development process, and evaluating your results.
  • Embed a screen-grabbed video of your software running.
  • Embed an animated GIF (or two) of your software.
  • Embed a still image of your software.
  • Embed some photos or scans of your notebook sketches.
  • Link to your code at the p5 Editor.
  • Test your blog post to make sure that all of the above embedded media appear correctly.

If you're having a problem, ask for help! 

Some more fun things to consider:

Face-Powered Shooter by Lingdong Huang (Built in Processing with FaceOSC, 2016)