Project 1: Flickr Scrape + Face Averaging

by jonathan @ 8:26 am 7 February 2012

For the first “real” project, I initially had a bit of trouble deciding what kinds of data I wanted to obtain. Initially I was pretty set on using Twitter and the Twitter 4j library for Processing. I was entranced by the thought of tapping into the real-time thoughts of Twitter users of Pittsburgh in relationship to the weather. I planned on creating a gradiation of color swatches by taking a photograph of the sky, averaging the color, and relating this color to the tweets during this time exploring the possibility that perhaps cloud breaks or the rare rays of sunshine would drastically affect the mood of local Pittsburghers. However, my idea lacked depth and the topic of weather was pretty much off limits for this project.

Therefore, I sought to explore another form of data visualization: face averaging. From the very beginning, I was somewhat obsessed with the concept of using averaging as a means of abstracting data whether it be the color of the sky or people’s faces. As my final idea, I sought to plug ambiguous words such as “success” and “money” into Flickr, scrape the resulting images, toss the images through Face Tracker  in OpenFrameworks, and finally average the faces in MatLab.

I’ll admit, for a novice Processing user, diving into OpenFrameworks, Face Tracker, and Flickr API simultaneously was quite daunting. I knew what I wanted to do and the general means of acheiving the goal, but the details escaped me. I had never really investigated the C++ programming language before and I wasn’t really sure how to implement the Flickr API.


Regardless I dove head first, writing (or rather copying) a Flickr scraper from the Processing forums.

Once with my images, I headed into OpenFrameworks and immediately running into headaches with adding Addons in XCode 3 (which Max Hawkins assured me would have disappeared had I already installed XCode 4). Anyways logically, I knew I had to interrupt the video feed into Face Tracker with my own images, find where the points were being drawn and store those points in a .txt I could access later. It’s a lot easier than it sounds.

Every error that would appear, I would tried to decode it by either copy and pasting it into Google only to be greeted with more mumbo jumbo I didn’t understand, or I would constantly pester Alex Wolfe to lend me a helping hand (which graciously she did). Mostly, however, I struggled on my own as I really desired to at least understand the basic foundation of C++ and Openframworks.

There were essentially 3 major parts I had trouble with: reading the .jpg’s sequentially, getting Face Tracker to read the images, and outputting the plotted points to a .txt file. I ran into a numerous amount of logic errors, other errors that came and went… Essentially it was a hack job of me cutting and pasting, typing in random classes and types into the .h file and throwing them in the main .cpp file with the hopes that it would do something that I would want without making XCode too mad. Most of the time, I was frustrated by my lack of knowledge of the C++ language, severly hindering how much I could figure out on my own.

Finally, at 2:00am this morning, Alex and I, (mostly Alex) were able to wrangle Face Tracker to do our bidding and export everything very nicely. Unfortunately, once I threw my images into MatLab, things went to hell…again. I had anticipated this somewhat beforehand, but I really did not know how to work around it: Face Tracker is not wholly accurate, meaning 75% of the time, it would not filter images correctly and find faces randomly in the frames or fail to line up with the faces in the image in the first place. Of course this meant that the averaged face would look nothing remotely human or even like anything at all.


1 Comment

  1. ========================================
    Jonathan Ota: Flickr Scrape + Face Averaging

    The face of success is… a Cue Ball?
    Where you working with Alex? Did she show a little of your project? If so, maybe you should have presented together, or at least back to back

    nice project overview in the first image

    Great design + walkthrough+1

    They both look surprisingly Childish / soft.
    Did he not realize before this presentation that those looked identical? Because they are crazy similar.
    Hard to get perspective, some time :P

    Good to explain your issues and that you still put together the examples composites by hand, good to illustrate the idea and then make it work later

    You really know how to present your work.
    ^^indeed, excellent print layout/grid work and generally good typography
    ^^And just great process explanation

    did you do horizontal mirroring on the terrorists so they were looking in the same direction before compositing?

    nice concept, unfortunate that the images are so similar when overlaid. < - i disagree, makes it more interesting. (agree) well, if the intention was to have something representative of each of the different terms then this isn't really the same, however it does seem like a good foundation for further exploration, esp with all the work/process devloped to achieve hte goal.I think it's an amusing social commentarysaowh. as in, is there really anything that differentiates us/the "good guys" from the terrorists? haha, a bit heavy handed poster. “america as face of Terror” Check out the work of Nancy Burson:
    In particular, her piece Warhead (1982) which is one of her early composites.

    According to what?? Where is the data/images?
    ^^ Agreed, where do you get the associations from? Flickr tags or search queries?

    I think more than the face I would be interested in backgrounds. A lot of government officials have very scripted backgrounds. And I wonder if there is interesting data there. I think its commendable : I think the journey is sometimes crazier than the final product and just as imp imo.

    Great idea. I’m curious: did you curate these images by hand, or have some algorithm? I feel like the impact is lost if you just went through and cherry-picked Republicans out of a Flickr search then if you just pulled images according to some predetermined criteria. Even if you’re building it by hand, I think choosing the first 7 images or whatever for some search term from Flickr is fine.

    great idea!- i wonder if you could skip resizing issues by searching for images of all the same size

    JO, Kudos for fearlessly tackling the world of multiple interfaces. Ballsy for a two week project. The output, manual or not was disturbing and thought provoking.
    **I’m jelly
    Great idea! It is interesting to see what averaging of different types of images referring to the same search can reveal. Awesome job of combining all the different interfaces.

    Nice design
    **love the use of so many programming languages to bring this all together.

    Love the fonts!!

    love the use of faces!

    interesting results

    Really funny that the face of success and terrorism are so close

    I don’t know how I feel about the overlays of photos one on top of the other. What if you can identify specific traits and focus on them? “The eyes of terror?” “the nose of terror”? “The mouth of freedom”?
    It will allow us to focus a bit more on something specific.
    *** Yeah. I feel that it ends up looking like a bad photoshop job.

This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License.
(c) 2023 Interactive Art and Computational Design, Spring 2012 | powered by WordPress with Barecity