SKETCH-A-LEVEL :: SOME PROCESS

THE ORIGINAL IDEA
The original idea for this project was that an individual could place a piece of paper on a table, draw on it, and then have characters projected on it. The user could interact with these characters using their finger, a game controller, or some other means. The idea seemed simple enough, however there were quite a few problems I encountered when trying to execute the project.

ISSUES WITH ORIGINAL IDEA / EVOLUTION OF THE PROJECT
The setup would need to be large, similiar to a Reactable table. This would be expensive, and as I would soon find out, pretty unnecessary. Seeing as only a small portion of the glass surface would be active for imagery and interaction, there really wasn’t a point in creating a large rig. In addition, such a setup needs a short throw projector. I lacked such a projector and had to make use of an old projector from studio. This projector had poor quality and a very long throw. In addition, I had to bounce the projection off a mirror to get an accurate size on the glass which ultimately led to the rig being even larger. The end product wasn’t favorable either as the characters projected were very fuzzy.

Finger tracking was so-so. I intially was using a Kinect with reading glasses to enable me to track a finger at a close range (within 2 feet). Even with the glasses, however, I was unable to get close enough to the image. This wouldn’t have been a problem if I didn’t need the data from my RGB camera as well. Confusing? Okay, well basically I needed the depth image for blob detection/finger tracking and I needed the RGB image for the creation of a collision map which the digital characters could interact with. Ultimately I ended up switching to blob detection using a regular webcam, but that proved to finicky as the lighting had to be just right. In order to get the lighting correct (so it could counter the light from the projector), I needed to mount lights close to the sketch or close to the camera. This made for an even more complex rig.

That’s when I started thinking about mounting the camera AND the projector underneath the table. Then I would use clear glass instead of frosted glass. Vellum instead of paper. And lastly I would use fiducial markers for finger tracking. This probably would have worked, but I would still need a better projector and this ultimately would be expensive.

That’s when it was suggested that I make an application for a touchpad/tablet. I had made simple applications in the past and I figured this would be a good excuse to try and get better at it. Ergo, Sketch-a-Level for Android.

And while I thought my issues would pretty much end there, I actually encountered quite a few problems I had never encountered before and therefore had no idea how to fix. I struggled primarily with “out of memory” errors, which I later partially resolved by managing my images better. I also ran into a lot of problems with frame rate and character control. Alas, however, I did in the end have a working code which I am relatively happy with.