Spencer Barton

28 Mar 2014

Looking Glass: Idea and Update

I want to explore non-linear storytelling. The reader will guide a character through the story using a display. The character will know where it is on the page and behave accordingly.

The user will control a see-through display that will have the character displayed. The display will know where it is over the page so that the character can be animated to interact with the page.

2014-03-28 13.44.03

OLED Diplay Tests

I got animations working on the OLED display. More details on this piece of hardware..

2014-03-24 16.02.02

In this case the character is the caterpillar from The Very Hungry Caterpillar by Eric Carle.

Moving Forward

The key details at this point are:

  • Localization – I’ve got a few ideas
    • Capacitive plate under a page with learning
    • Microphones attached to page to listen for movement with learning
    • Camera mounted above to track
    • IR grid/ id tags on the page
    • Look at color beneath display
  • The story – I can use an existing story or create a new one. I am able to develop a story idea but would not be able to create a story graphically. I need to talk to others in the class and see if anyone would be interested in working on this part of the project with me.
  • Building the module piece – The current OLED display is flimsy and uninteresting. The reader deserves to hold a more interesting object. I need to create an object that pertains to the reading process that will contain the display, a processor, batteries and sensors. As a result it will be a nice hand-sized object. As a default I 3D print an object with a form similar to a computer mouse.

I am currently exploring the color detection case. With 2 or more color sensors it would be potentially possible to know exact location. That said the sensors are noisy so detection will likely occur based on regions. For example saying the the caterpillar is over the leaf or over a blank part of the page. The key advantage of the color sensors is that the display can become a completely independent object. All of the other ideas involve modifying the book or setting-up external hardware. The OLED display enable the reader to still engage with the book in a semi-natural manner. External hardware makes the reading situation more contrived and forces the reader to conform to my project set-up.

This project may be nice for the Pittsburgh Children’s Museum if ruggedized enough.