Category: ManifestoReading


“3. The Critical Engineer deconstructs and incites suspicion of rich user experiences.”

I found this tenet of the Critical Engineering Manifesto curious, since rich user experiences are typically thought to be solely positive. Upon further thought, very rich user interface can disguise underlying problems. A very basic example would be purchasing some sort of commodity, say headphones. People might want to buy really edgy and well designed headphones even if it’s overpriced and the sound quality isn’t as good as alternatives. This happens with the way almost anything is packaged. When strong user interface and convenience intersect, the better alternative solutions and problematic engineering can be swept under the rug. A basic (perhaps opinionated) example would be the popularity of Venmo. Linking your banking info to this mobile app often does raise red flags due to its convenience, despite the fact that it’s not even that much of a hassle to pay someone back in cash. This surely extends to issues whose consequences go beyond the individual.


The tenet I chose is the one where the critical engineer doesn’t just marvel at a new technology because it is new and combines cool elements of technology. The tenet says that the critical engineer looks beyond how their work is implemented to see how it will actually have an impact, more so the critical engineer digs into specificity of the impact of their work. An example of this is the advent of the social media platform. When Facebook, Twitter, Snapchat, etcetera became popular it was initially conceived of as a way to share light-hearted photos, jokes, and stories. However, the engineers didn’t really look in to the depth of what it means to be social. Being social means to sometimes be envious, which is why people who spend unhealthy amounts of time on social media are prone to depression. Being social means to compete to get the most friends, to ask people for money, to argue, and to ignore. But because the focus originally was on the technology and the ability, it wasn’t until years later where the apps were refined to account for things like fraud, hate speech, suicide posts.


It is very difficult to choose just one tenant of this Critical Engineering Manifesto. Many of the tenants are interrelated, and they feed directly into much of the reading I’ve been doing recently on the human-machine entanglement.

Therefore, I pick three tenants, which I believe to be highly interrelated:

1. The Critical Engineer considers any technology depended upon to be both a challenge and a threat. The greater the dependence on a technology the greater the need to study and expose its inner workings, regardless of ownership or legal provision.

2. The Critical Engineer raises awareness that with each technological advance our techno-political literacy is challenged.

9. The Critical Engineer notes that written code expands into social and psychological realms, regulating behaviour between people and the machines they interact with. By understanding this, the Critical Engineer seeks to reconstruct user-constraints and social action through means of digital excavation.

These concepts all hinge on the power and value hierarchy wielded by those who create the “black box” around new technological developments. We have seen, with vivid and brutal clarity, what happens when we depend on a technology and allow it to “regulate our behavior” – remaining within its tightly controlled constructs, and not the questioning the legitimacy of this dependency. The “echo chamber” around social media, the propagation of fake news, the threats to cybersecurity – all of these relate to our “techno-political literacy.” When we depend on a technology, we render ourselves vulnerable to its exploitation. As the perpetual “forward march of progress” in technology continues, we are challenged to understand the new developments as they affect our liberty, communication, and access to information. The more these technological developments remain underneath the “black box” veil, the more we must apply the tenants of the Critical Engineer, to “expose its inner workings,” and “reconstruct user-constraints and social action through means of digital excavation.”

Lucy Suchman articulates the importance of “Critical Technical Practice, in which attention to the rhetorics and technologies through which a field constructs its research objects becomes an integral part of its research practice.” – Suchman, Human-Machine Reconfigurations : Plans and Situated Actions, 2nd Edition, 2007. As Critical Engineers and Practitioners, we must be self-aware and critical of our own rhetoric surrounding the technology we develop and work with – thus continually unmasking the “black box” – and refusing to become the robots of our own design.


Clacker- Manifesto Reading

“ 4. The Critical Engineer looks beyond the “awe of implementation” to determine methods of influence and their specific effects.”

To me this means: Making something new is not enough. It is not enough to use computation and engineering in a impressive way if the impressive element is the fact that it works. It is not enough to take advantage of a naive audience and impress with the lowest level of criticality, while relying on the novelty of futuristic technology. Aiming for only the “wow” factor of a working prototype is not a sustainable practice and does not reflect the work of a critical engineer. A critical engineer is not looking for the short term glory of something that is impressive because it works. Instead a critical engineer looks past the short term and uses their ability to ask questions and make things with a purpose, not just becasue they can. This purpose is to think about the tools they are using in the context of the world, and more specifically in the context of their interests. Only by employing this kind of critical thinking can a critical engineer determine the way they want to influence others with their work, and learn from their work. Determining methods of influence and their effects requires a hypothesis and goal that can be evaluated after the completion of the work. This evaluation will be complex, where as the evaluation of a project that relies on the “awe of implementation” a binary question; does it work? 


6. The Critical Engineer expands “machine” to describe interrelationships encompassing devices, bodies, agents, forces and networks.

This tenent is connected to several others in the manifesto (namely 9 and 4) but most directly articulates the underlying idea that and single engineering work does not exist in a vacuum. There are always social, political, emotional, humanitarian, economic, and other conversations which influence and can be influenced by an engineer’s creation. This core idea, like many of the others, applies to any sort of manufactured culture (movies, books, toys, software, art…). A critical engineer will always recognize and strive to examine the nuanced forces that could be affected by a “machine,” to look beyond the inherent workings of any singular device at the broader ramifications of it within our culture. An example, which is always at the forefront of my mind, is virtual reality headsets. Often they are manufactured because they’re cool; students want to work at Oculus because the tech is new and interesting, the job pays well, but the creation of virtual worlds is endlessly important in how society evolves. If creators aren’t careful with the content, or if engineers aren’t considerate of the accessibility, virtual reality works can be a source of contention and division in society. Obviously there will always be upsides and downsides, societal speaking, to any “machine” but makers should be aware of them.


4. The Critical Engineer looks beyond the “awe of implementation” to determine methods of influence and their specific effects.

In today’s world, so much more time, energy, and money are spent on shiny new gadgets for people to buy before the newer, shinier one comes out. But what good are most of those gadgets doing for society? Not saying everything created has to be utilitarian, but being aware of what a new technology can do, and making a purposeful effort to curb negative social side effects is an important step in the right direction. Creating things that create meaningful interactions should be the goal more so than making new annoying MacBooks that don’t have USB ports because you know people will buy them anyway.


“5. The Critical Engineer recognises that each work of engineering engineers its user, proportional to that user’s dependency upon it.”

I think the tenet means that a work of engineering modifies the way the user thinks about the problem, so the more the user uses it, the more the user thinks in its way. I found this particularly interesting because I’m gradually becoming aware of the fact that I’m being engineered by so many things around me. I’m also thinking about common things that could have been engineered in another way and what would happen to us if they were.

One example would be the Processing language. After using Processing a lot for a semester, I begin thinking in Processing’s way. I think of the screen as a canvas, and lines and shapes are drawn onto it to make a frame. Even when I’m not using Processing, or when I’m just thinking about random ideas, (or when I’m thinking about things not even related to programming,) this mode of thinking sticks. Similarly, if I have been using d3 for the whole semester, I might have think of programing as data entering and exiting.


3. The Critical Engineer deconstructs and incites suspicion of rich user experiences.
from CE

“Any sufficiently advanced technology is indistinguishable from magic.” is the third of Arthur C. Clarke’s three laws. In many cases, the work of artists, designers, and others who deliver ‘human-experiences’ use technology to create this ‘rich user experience’ magic. Of course, those doing this art & design work are often their own engineers, doing a type of engineering themselves (even if they’re using art-engineering toolkits like openFrameworks or Processing.) If by extension we then say they should adopt the mindset of Critical Engineering, the point at which the show is over and the trick can be ‘revealed’ is one of contention.

I think the very existence of the open source software movement in conjunction with GitHub has shown that many artist-engineers are freely willing to share what they make and how they do it. Even companies like Disney and Microsoft reveal a great many of their tricks through their large research organizations which publish findings regularly.


The excerpt from “The Epic Struggle of the Internet of Things” read to me as… overly complicated and perhaps oddly unnecessary to me–as did the “Critical Engineering Manifesto”, but the latter delved into a depth of societal introspection that I merely could not grasp nor conjure enough interest. What I absorbed from the former, at least, was the hypothetical scenario of a consumer attempting to merge the utility of two seemingly unrelated products; oftentimes this assumes the objects to be of different technological time periods, which implies the desire to technologically advance the more mundane, archaic device of the two. I personally relate to the individuals who do not see the practical point in this mindset: with modern day technology naturally being comparatively impressive to the past, and constantly advancing to encompass more opportunities, there is constantly the rise of arbitrary explorations in how to make sometimes the most unnecessary things “more technologically advanced”. This can be the result of different factors, from pure curiosity to inherent human laziness. Though this “struggle of the internet of things” has potential to pave ways to new, useful, innovative inventions that are practical and reasonable, people believe, and I can sympathize, that plenty results are probably made purely “just because”, and with no real usable value. I have not dealt with this concept much at all, but perhaps a reflexive example I can come up with is the new Apple updates recently, for both the newest iPhone and Macbooks, both of which ignited anger and frustration in all my tech-savvy friends (much to my personal apathy as a non-Apple fanatic); from what I’ve heard, it feels like Apple has just been “upgrading” (debatable, haha), its specs of its newer products just for the sake of implementing some change, not really thinking about the practicality in actual use and potential feedback of its loyal customers. Although this isn’t combining two objects, this was the closest example I could probably relate to as someone who has never thought about, and is confirmed to be thoroughly uninterested, in this “struggle of the internet of things”.


7. The Critical Engineer observes the space between the production and consumption of technology. Acting rapidly to changes in this space, the Critical Engineer serves to expose moments of imbalance and deception.

In my own words — Critical engineers must be aware, understand, and take responsibility when creating a new innovation that will be provided to the general public. Critical engineers must strike a balance between the idea of new innovation, and natural human behavior when humans interact with new technologies.

I find this tenet interesting because it resonates with a lot of what we are taught in CMU’s School of Design. Until today, the main “manifesto” that most designers talk about is Dieter Ram’s 10 Principles for Good Design. I think there’s a lot of similar themes such as sustainability and responsibility that we have the ability to affect human behavior, especially for product designers who work with engineers.

CMU Design revamped their curriculum when I entered in 2014 to focus on Transition Design, or designing for sustainability. I came here thinking I’d learn how to make pretty, aesthetic things that people would buy because they looked pretty. NOW I realize the responsibility we have as makers to think about the magnitude of our decisions and how we can have a real influence in how people live their lives. While new technology and the “Internet of Things” sounds like cool stuff, are conversations and decisions being made about user needs, important intentions, and what type of future we want to live in?



Catlu – Manifesto

One tenet of the manifesto I found interesting was tenet number 1. The tenet basically says that every piece of technology we depend on must be considered both a “challenge and a threat.” Because we depend on these objects, it is imperative that we know them inside and out, all their workings, so we can rise to challenges that may arise due to our dependency on them, perhaps even shake ourselves from their shackles, and also to be prepared for the event of their failure or effects. This should be done with all technology regardless of “ownership or legal provision.” To me, this tenet is extremely important as we continue in the technological era. Technology exists so much around us that it’s not something a lot of us think too much about anymore. We assume that all our commodities will continue working forever. This dependency combined with our mindlessness could end in catastrophe. Therefore I agree that it’s true that a critical engineer must not only think about the great effects that a new or old invention may have if it exists, but also the negative effects of its existence along with the effects of its absence after a prolonged period. If when the internet, a year after it was first invented suddenly crashed and disappeared, it would maybe have been inconvenient, but if the internet crashed tomorrow, there would be a global crisis. Panic would spread as information would be lost, communication down, and a massive amount of commodities the internet provides that we simply don’t know how to live without. I know some people that can’t even get around without Google Maps. For this perhaps eventual crisis, I don’t know if we have a backup. I don’t know what the damage could be, and that is terrifying. In order to be able to bring in new technology, we have to first be the critical engineer, and look further than the technology itself so we can gauge the cost of dependency and the perhaps unexpected costs of its existence.

Xastol – ManifestoReading

From the manifesto reading, Tenet #1 is the most compelling to me. Tenet #1 says that the Critical Engineer looks at technology and its effects on the well-being of society. If this technology proves to be a possible threat to said society, then the Critical Engineer’s job is to evaluate the threat and propose a change/solution regardless of any legal protections. I think this is interesting because a Critical Engineer could be anyone in society. I feel like this tenet says it’s up to the people that make up the social structure to determine if the technology is a possible threat and whether to abolish, change, or keep it.

An example of this tenet is obvious in intellectual property laws. Although the entire point of intellectual property laws are to give ownership to technology/work, there are cases where information/technology is seen as public domain and deemed imperative that citizens have access.


The Cyber


So we had to get very, very tough on cyber and cyber warfare. It is a huge problem. I have a son—he’s 10 years old. He has computers. He is so good with these computers. It’s unbelievable. The security aspect of cyber is very, very tough. And maybe, it’s hardly doable. But I will say, we are not doing the job we should be doing.

2. The Critical Engineer raises awareness that with each technological advance our techno-political literacy is challenged.

This second point in the Critical Engineer’s Manifesto was the most thought-provoking for me, because it underscores a point that goes so often unacknowledged in discussions of new technologies: Not only do advancements in technology pull us father from understanding the mechanics of the new technology, but abstract and obscure our ability to discuss the ethical, political, and social implications of their implementation.

One of the most significant examples of this comes in the form of politicians discussing the relatively recent phenomena of cyber-warfare. Listening to almost any politician discuss their opinions or policy surrounding cyber-warfare, it becomes apparent that usually lack an even shaky understanding of cryptography, hacking techniques, much less how the internet fundamentally works. But the thing is, you can’t blame them! The average citizen doesn’t have any of this knowledge either, hardly anyone does except discipline experts, and even then, you would need a panel of specialists to explain every part of it.

While these layers of technological abstraction, buildings blocks on building blocks that afford us everything that the internet offers us on an essentially hourly basis, it makes us as a society, policy makers and citizens, largely unable to have any sort of informed or sophisticated conversation about the ethics, limitations, and boundaries of these systems. Arguments get boiled down to meaningless phrases and rhetoric that lack any real substance.

While I don’t exactly see a solution to this growing divide between knowledge of tech systems and legislation relating to them, I think those who are working and studying in the sphere of tech need to be much more firmly brought into conversations about ethics, understanding the immense power and scale of their field.

Jaqaur – Manifesto

One part of the manifesto that stuck out to me was tenet number 4: The Critical Engineer looks beyond the “awe of implementation” to determine methods of influence and their specific effects. This is basically saying that it’s important to consider exactly why you are making the choices you are, and why you are developing the things you are, and if the answer is “because we can,” maybe you should reconsider. It reminded me of Jurassic Park when Ian says “Your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should.” Just because something can be developed doesn’t mean its worth the time and resources, even if it would be really impressive or cool. Ultimately, the point of engineering is to improve lives, not to impress others.


Of course the Critical Engineer manifesto starts counting at 0.

As a whole I think the manifesto is very similar to the unspoken oath we take as designers. We look to other disciplines and fields to learn and re-apply their methods. At CMU it is mandatory for all designers to take classes such as Intro to Psychology, Systems, Cultures, and Futures. We are constantly taught to think about how design works with the user, the people included in the production of the design, and the society and culture it will live in. We are taught to think of unintended uses and unpredicted malicious affordances. We are always looking past the shiny and pretty, and looking for true innovation and disruption. I think I really resonated withe piece and that with a bit of change in language, it could easily be the designer’s manifesto. I guess this piece really showed the that critical and engineer and the conscious designer are sisters in a sense.

The only piece that I felt a bit of confusion and that I needed to delve further into was the last. 10. The Critical Engineer considers the exploit to be the most desirable form of exposure.

After struggling to get past the word exploit, I looked it up to be sure I was interpreting it correctly.

  1. a bold or daring feat: the most heroic and secretive exploits of the war.
  2. a software tool designed to take advantage of a flaw in a computer system, typically for malicious purposes such as installing malware: if someone you don’t know tweets you a link, it’s either spam, an exploit, or probably both.

I just think that the two definitions give different, but equally interesting meanings to the final piece of the manifesto. The critical engineer finds that the greatest achievement or challenge is the most preferred way of being viewed. It could also be interpreted as the critical engineer the flaw or the bug the best way to objectively see the system. Is it our greatest flaw or our greatest achievement that will show off our truest self?

After showing the manifesto with my peer, an electrical computer engineer major, I asked for his opinion. He thought the piece was elegant and incredibly accurate, and mentioned that he thought that all engineers should be required to read it. I asked which was his favourite, and he too picked the 10th tenant. He provided an alternative, and probably more accurate, interpretation. He explained that getting hacked would be the greatest form of flattery. If your work is getting hacked then it suggests that the work is big enough or important enough for someone else to spend time trying to break it. In essence, the engineer believes that if their work is being exploited, this means their work is good enough to be worth trying to exploit.



I believe that among all the engineering tenets, #9 is the most important, relevant and (in my opinion) impactful. I say this because the code in anything defines how digital technology works, and digital technology is by far the biggest, most influential, life-impacting and ever-present form of technology, innovation and invention since the start of mankind.

Its very important to maintain a balance between human and machine interaction. If one confuses the other the balance will be broken. This tenet is saying how engineers should not write code purely for function but for the response and perception by people that interact with it. They should delve into psychological and social realms as the tenet says, and should work to create the most immersive digital experience possible without ruining the harmony with the physical world. There are a few large corporations that seek to create a seamless communication between the two, “perfecting” the experience, such as Apple (during the years Steve Jobs was in charge). The company has always aimed to create a simple-to use (to minimize frustration) yet powerful operating system experience on all of the devices they own, from mobile devices like the iPhone to desktops and laptops. Not only do they strive to minimize the friction between person and machine interaction, they have developed a tight and invisible connection between all of their own machines, creating a machine-to-machine environment that creates an even better experience as you switch from one to the other without problem.


The Critical Engineer recognizes that each work of engineering engineers its user, proportional to that user’s dependency upon it.


This tenant means that even though things are made for people to use, which implies that they should bend to human wants and needs, that is not the case. People adapt to their technology, which is constructed for them, but also adapts in response to how people use it, and again people adapt to this change. Basically, critical engineering is a never ending cycle that alters the person and the engineered object over and over again. I think this is interesting cause it shows how nothing designed for people can truly ever be ‘finished’, not can it really be optimized since people are constantly adapting to their environments (which include the tech itself) and because of personal tastes and the difference between people that are either bridged or expanded.



Lumar- Reading-Response #08: Two Readings about Things

“1. The Critical Engineer considers any technology depended upon to be both a challenge and a threat. The greater the dependence on a technology the greater the need to study and expose its inner workings, regardless of ownership or legal provision.”

“5. The Critical Engineer recognises that each work of engineering engineers its user, proportional to that user’s dependency upon it.”

I am looking forward to the design lecture tomorrow. I am both excited and incredibly wary of the booming rise of the internet of things. I can imagine it can very easily be another subject that would go perfectly as an expansion of the documentary “death by Design”. The documentary “Death by Design” explores the question –

“What is the cost of our digital dependency?”

It uncovers a global story of damaged lives, environmental destruction, and devices that are designed to die.

With the engineering principles, I find #1 something that’s emphasized in my design studios. When designers include ‘fancy’ futuristic tech as solutions in their concept pitches, the biggest warning is always that the designer has to be able to consider carefully how to make the technology work for people, finding the gaps within the system that the tech could fulfill rather than making gaps to the existing system to necessitate a convoluted solution. I like to liken the use of technology that one doesn’t fully understand as a design solution problem to the federal government pouring money into a flopping program; it doesn’t really help. Well….I mean, it does. Money will inherently make the wheels spin faster, but the amount of money certainly isn’t proportional to the net benefit. The money simply isn’t being used effectively. Technology, if not understood well, is much the same.

For number 5, I agree! Tools shape you, you shape tools!






That’s a quote directly from Graphic #37, a graphic design magazine that introduces computation as a medium for graphic design.

An analog example of this sentiment would be in the evolution of symbols. When a designer first makes a symbol for an object, it tends to be more literal and representative. But as the public gets used to this association, the next generation of designers would redesign the symbol to be a simplified version of the previous. If this second symbol was used at the very beginning, it might not be nearly as effective. (think of the symbols for phone)



kander – manifesto

It seemed that most of the “critical engineer” points were concerned with considering the implications of technological developments on the world. They were mainly about acknowledging that technology doesn’t exist in a bubble, and implied things like ethics and impact shouldn’t be disregard for the sake of “progress.” I found item #6 particularly interesting:

“The Critical Engineer expands “machine” to describe interrelationships encompassing devices, bodies, agents, forces and networks.”

In other words, technology shouldn’t be exclusively defined to be an item comprised of solely physical elements/hardware. The definition of technology and machines needs to take into account the space in which the item occupies in the world — an iPhone would have very different implications if it was an item owned by only the wealthiest, or was something that everyone in the world had access to. To continue with the iPhone example, it’s uses would be much different if it couldn’t connect to the Internet.

This last example brings up the interesting idea that part of the iPhone’s invention is the Internet. If we extend this principle — that all technologies that interact with other innovations include said innovations in their own makeup — then we begin to see modern technology not as a set of individual components, but a web of connected ideas and devices that build upon each other.



Drewch – ManifestoReading

2. The Critical Engineer raises awareness that with each technological advance our techno-political literacy is challenged.

This tenet of the manifesto is interesting to me because I have seen its effects in action. The stereotype of the computer-illiterate parent/grandparent is based on this. The huge, sweeping adoption and evolution of computers left generations of individuals, born and raised during technologically simpler times, in the dust. Then came phones and social media, and warnings of becoming slaves to the instruments. This trend isn’t just limited to electronic inventions, however. The first time the printing press was adopted, it was violently opposed because of how it would “create forgetfulness in the learners’ souls, because they will not use their memories,” or something like that. Even further back, writing was treated the same way. Whatever the next step may be, be it AR or AI, I’m ready to hear all about the moral and social outcries.

Deliverables 08 (Due 11/11)

This assignment has three parts: Some readings, a Looking Outwards, and a software project. Please note that these deliverables have different due dates: 

  • Part A. Reading-Response #08: Two Readings about Things, due Monday 11/14
  • Part B. Looking Outwards #08: On Physical Computing, due Monday 11/14
  • Part C. Software for a Skeleton (Computation + Mocap), due Friday 11/11
    • Ten Creative Opportunities
    • Technical Options & Links
    • Summary of Deliverables

Part A. Reading-Response #08: Two Readings about Things

This is intended as a very brief reading/response assignment, whose purpose is to introduce some vocabulary and perspective on “critical making” and the “internet of things”. You are asked to read two very brief statements.
Due Monday, November 14.

Please read the following one-page excerpt from Bruce Sterling’s “Epic Struggle for the Internet of Things”:

Please (also) read the one-page “Critical Engineering Manifesto” (2011) by Julian Oliver, Gordan Savičić, and Danja Vasiliev. Now,

  • Select one of the tenets of the manifesto that you find interesting.
  • In a brief blog post of 100-150 words, re-explain it in your own words, and explain what you found interesting about it. If possible, provide an example, real or hypothetical, which illustrates the proposition.
  • Label your blog post with the Category, ManifestoReading, and title it nickname-manifesto.

Part B. Looking Outwards #08: Physical Computing

This LookingOutwards assignment is concerned with physical computing and tangible interaction design. As part of this Looking Outwards, you are strongly strongly encouraged to attend the public lecture of Hiroshi Ishii on Thursday, November 10 at 5pm in McConomy Auditorium. (Chinese food will be served afterwards in the STUDIO.)
Due Monday, November 14.


Here are some links you are welcome to explore for your Looking Outwards assignment:

Physical computing projects:

Arduino (specific) projects:

Please categorize your Looking Outwards with the WordPress Category, LookingOutwards08, and title your blog post nickname-lookingoutwards08.

Part C. Software for a Skeleton

For this project, you are asked to write software which
creatively interprets, or responds to, the actions of the body.

You will develop a computational treatment for motion-capture data. Ideally, both your treatment, and your motion-capture data, will be ‘tightly coupled’ to each other: The treatment will be designed for specific motion-capture data, and the motion-capture data will be intentionally selected or performed for your specific treatment.

Code templates for Processing, three.js and openFrameworks are here.
Due Friday, November 11.

Ten Creative Opportunities

It’s important to emphasize that you have a multitude of creative options — well beyond, or alternative to, the initial concept of a “decorated skeleton”. The following ten suggestions, which are by no means comprehensive, are intended to prompt you to appreciate the breadth of the conceptual space you may explore. In all cases, be prepared to justify your decisions.

  1. You may work in real-time (interactive), or off-line (animation). You may choose to develop a piece of interactive real-time software, which treats the mocap file as a proxy for data from a live user (as in Setsuyakurotaki, by Zach Lieberman + Rhizomatiks, shown above in use by live DJs). Or you may choose to develop a piece of custom animation software, which interprets the mocap file as an input to a lengthy rendering process process (as in Universal Everything’s Walking City, or Method Studios’ AICP Sponsor Reel).
  2. You may use more than one body. Your software doesn’t have to be limited to just one body. Instead, it could visualize the relationship (or create a relationship) between two or more bodies (as in Scott Snibbe’s Boundary Functions or ). It could visualize or respond to a duet, trio or crowd of people.
  3. You may focus on just part of the body. Your software doesn’t need to respond to the entire body; it could focus on interpreting just a single part of the body (as in Theo Watson & Emily Gobeille’s prototype for Puppet Parade, which responds to a single arm).
  4. You may focus on how an environment is affected by the body. Your software doesn’t have to re-skin or visualize the body. Instead, you can develop an environment that is affected by the movements of the body (as in Theo & Emily’s Weather Worlds).
  5. You may position your ‘camera’ anywhere — including a first-person POV, or with a (user-driven) VR POV. Just because your performance was recorded from a sensor “in front” of you, this does not mean your mocap data must be viewed from the same point of view. Consider displaying your figure in the round, from above, below, or even from the POV of the body itself. (Check out the camera() function in Processing, or the PerspectiveCamera object in three.js, for more ideas. If you’re using three.js, you could also try a WebVR build for Google cardboard.)
  6. You may work in 3D or 2D. Although your mocap data represents three-dimensional coordinates, you don’t have to make a 3D scene; for example, you could use your mocap to control an assemblage of 2D shapes. You could even use your body to control two-dimensional typography. (Helpful Processing commands like screenX() and screenY() , or unprojectVector() in three.js, allow you to easily compute the 2D coordinates of a perspectivally-projected 3D point.)
  7. You may control the behavior of something non-human. Just because your data was captured from a human, doesn’t mean you must control a human. Consider using your mocap data to puppeteer an animal, monster, plant, or even a non-living object (as in this research on “animating non-humanoid characters with human motion data” from Disney Research).
  8. You may record mocap data yourself, or you can use data from an online source. If you’re recording the data yourself, feel free to record a friend who is a performer — perhaps a musician, actor, or athlete. Alternatively, feel free to use data from an online archive or commercial vendor. You may also combine these different sources; for example, you could combine your own awkward performance, with a group of professional backup dancers.
  9. You can make software which is analytic or expressive. You are asked to make a piece of software which interprets the actions of the human body. While some of your peers may choose to develop a character animation or interactive software mirror, you might instead elect to create “information visualization” software that presents an analysis of the body’s joints over time. Your software could present comparisons different people making similar movements, or could track the accelerations of movements by a violinist.
  10. You may use sound. Feel free to play back sound which is synchronized with your motion capture files. This might be the performer’s speech, or music to which they are dancing, etc. (Check out the Processing Sound Library to play simple sounds, or the PositionalAudio class in three.js, which has the ability to play sounds using 3D-spatialization.)

Technical Options & Resources

Code templates for loading and displaying motion capture files in the BVH format have been provided for you in Processing (Java), openFrameworks (C++), and three.js (a JavaScript library for high-quality OpenGL 3D graphics in the browser). You can find these templates here:

As an alternative to the above, you are permitted to use Maya (with its internal Python scripting language), or Unity3D for this project. Kindly note, however, that the professor and TA cannot support these alternative environments. If you use them, you should be prepared to work independently. For Python in Maya, please this tutorial, this tutorial, and this video.

For this project, it is assumed that you will record or reuse a motion capture file in the BVH format. (If you are working in Maya or Unity, you may prefer to use the FBX format.) We have purchased a copy of Brekel Pro Body v2 for you to use to record motion capture files, and we have installed it on a PC in the STUDIO; it can record Kinect v2 data into these various mocap formats.

Our Three.js demo (included in BVH example code):


Our Processing demo (included in BVH example code):


Summary of Deliverables

Here’s what’s expected for this assignment.

  • Review some of the treatments of motion-capture data which people have developed, whether for realtime interactions or for offline animations, in our lecture notes from Friday 11/4.
  • Sketch first! Draw some ideas.
  • Make or find a motion capture recording. Be sure to record a couple takes. Keep in mind that you may wish to re-record your performance later, once your software is finished.
  • Develop a program that creatively interprets, or responds to, the changing performance of a body as recorded in your motion-capture data. (If you feel like trying three.js, check out their demos and examples.)
  • Create a blog post on this site to hold the media below.
  • Title your blog post, nickname-mocap, and give your blog post the WordPress Category, Mocap.
  • Write a narrative of 150-200 words describing your development process, and evaluating your results. Include some information about your inspirations, if any.
  • Embed a screengrabbed video of your software running (if it is designed to run in realtime). If your software runs “offline” (non-realtime), as in an animation, render out a video and embed that.
  • Upload an animated GIF of your software. It can be brief (3-5 seconds).
  • Upload a still image of your software.
  • Upload some photos or scans of your notebook sketches.
  • Embed your code (using the WP-Syntax WordPress plugin to format your JavaScript and/or other code), and include a link to your code on Github.
  • Test your blog post to make sure that all of the above embedded media appear correctly. If you’re having a problem, ask for help.

Good luck!

Comments Off on Deliverables 08 (Due 11/11) Posted in