After trying for hours to get ofMoviePlayer to do a simple loadmovie case switch, I found on the forums that there is a problem with the 62 video player.

Therefore for generative video cuts, I have to rollback to OF61 video player

http://forum.openframeworks.cc/index.php?topic=5729.0

A pdf! Containing a large number of screenshots

as

This is my initial auto thresholding. You can see the depth histogram on the bottom.

This one uses ofxControlPanel to allow for adjustment of some settings and a video library.

a

I’m working on correctly calculating the bounce direction and velocity. Using OpenNI, I track the y position of the base of the neck. With a small time delta, I look at the difference between the two positions to get a reasonable approximation of both direction and velocity. After introducing an actual trampoline, I had to significantly reduce my periodic sampling because I could perform a full bounce and return to a position close to the source position before the sample would be taken. I haven’t mapped velocity to sidescrolling action, so there isn’t much to see, but here’s a picture of a trampoline:

Bounce bounce bounce.

Over this weekend, I’ve succeeded in finishing a basic build of a game using the Magrathea system. The game is for any number of players, who build a landscape so that the A.I. controlled character (shown above) can walk around collecting flowers, and avoiding trolls.

Building this game involved implementing a lot of features – automated systems for moving sprites, keeping track of what sprites existed, etc, and finding modular, expandable ways to do this was a lot of my recent work. The sprites can now move around the world, avoid stepping into water, display and scale properly, etc.

The design of the game is very simple right now. The hero is extremely dumb – he basically can only try to step towards the flower. He’s smart enough not to walk into water, and not to walk up or down too-steep cliffs, but not to find another path. The troll is pretty dumb too, as he can only step towards the hero (though he can track a moving hero).

I’m not keeping track of any sort of score now (though I could keep track of how many flowers you’ve collected, or make that effect the world), because I’m concerned about the game eclipsing the rest of the tool, and I think that’s what I’m struggling with now.

Basically, I’m nervous that the ‘game’ isn’t really compelling enough, and that it’s driven the focus away from the fun, interesting part of the project (building terrain) and pushed it into another direction (waiting as some dumb asshole sprite walks across your arm).

That said, I do think watching the sprites move around and grab stuff is fun. But the enemies are too difficult to deal with reliably, and the hero a little too dumb to trust to do anything correct, requiring too much constant babysitting.

I also realize that I’ve been super-involved with this for the last 72 hours, so this is totally the time when I need feedback. I think the work I’ve done has gone to good use – I’ve learned how to code behaviors, display sprites better, smooth their movement, ensure they are added onto existing terrain, etc. What I’m trying to decide now is if I should continue in the direction of refining this gameplay, or make it into more of a sandbox. Here’s the theoretical design of how that could happen (Note, all of the framework for this has been implemented, so doing this would require mostly the creation of more graphical elements).:

The user(s) builds terrain, which is populated with a few (3? 4?) characters who wander around until something (food, a flower, eachother) catches their attention. When this happens, a thought balloon pops up over their head (or as part of a GUI above the terrain? this would obscure less of the action) indicating their desire for that thing, and they start (dumbly) moving towards it. When they get to it, they do a short animation. Perhaps they permanently affect the world (pick up a flower then scatter seeds, growing more flowers?)

This may sound very dangerous or like I’m in a crisis, but what I’ve developed right now is essentially what’s described above, but with one character, one item, and the presence of a malicious element (the troll), so this path would really just be an extension of what I’ve done, but in a different direction than the game.

I’m pretty pleased with my progress, and feel that with feedback, I’ll be able to decide which direction to go in. If people want to playtest, please let me know!!

(also, i realize some sprites (THE HOUSE) are not totally in the same world as everything else yet, it’s a placeholder/an experiment)

screen shots (click for full):

I am trying to work along my plan and schedule for this final project. Ideally I will finish 3~5 fully functioned drum bots to demonstrate potential combination of music and algorithm.

What I have done:

Spend some time redesigned my own Arduino-compatible circuit board, which is less smaller than the original one and with all sockets and electronic parts on. (1 x 1 inch)

Ordered all parts from Digikey last Wed.

Tried a lot. But still remains some problems:

piezo sensor is really fragile and not stable.

Solenoids are either too weak or eat too much current & voltage.

Batteries are always the problem.

Circuit board never shipped.

Sounds effect.

Expectation:

In Thursday have a working demo with 3~5 drumboxs

Software is hard, hardware is HARDER!

start here

1. color – sepia, minimal
2. lineart – edges are important
3. trails – particles or such

Live Feed, 2-toned

Edge detection, understood and working stand-alone

Combining? Broke it. Inefficient keyboard banging to no means, eyes bleeding. (This is a overlay Photoshopped to demonstrate the somewhat desired effect.)

Next: oFlow, learning from this good example found at openprocessing…

Scoping… Make this into a part in series of learning bits of coding.
Combos in mind include:
1. 2 color + trails
2. 8bit/block people + springs
3. Ballpit people / colorblind test people + boids

Display may be something like this..

Link to demo

Original idea

• Users each control a blob
• Blobs can interact with each other
• Playstation Home, but with blobs

New Idea

What’s been done

My project lives as it grows at http://markshuster.com/iacd/4/

Updates soon.

We’re not going to have a polished, pretty, finished version of mr. BEHEMOTH kicking around the STUDIO until the show on the 28th, but we’ve managed to make some progress despite the universe’s best efforts to keep us down. With Golan’s help, Asa wrangled the mighty DOT PRODUCT, “an algebraic operation that takes two equal-length sequences of numbers and returns a single number obtained by multiplying corresponding entries and then summing those products.”

In layman’s terms, our program now reads the angles between sets of skeleton joints in 3 dimensions, instead of 2. This is preferable for a multitude of reasons; mostly for the sheer intuitive nature of the project, which was practically nonexistent in the first iteration: Mr. BEHEMOTH moved as nicely as he did because Asa & I understood the precise way to stand in order to make it not look like our puppet was having a seizure.

Now, a person can be much more relaxed when operating the puppet, and the results will be much more fluid.

Caitlin’s in charge of making the BEHEMOTH more dynamic, & created a sound trigger for the star-vomit from our update before last. Now, users can ROAR (or sing, or talk loudly) into a microphone, and the stream of stars will either be tiny or RIDICULOUS depending on the volume and length of the noise.

+ =

Golan is trying to talk Caitlin into using this same feature to make the BEHEMOTH poop. Caitlin really doesn’t wanna make it poop. Your thoughts on poop, class?

C is also still kicking around with Box2D to add some springy, physicsy elements to the puppet, but these need some work, as Caitlin reads slowly and is still picking apart Shiffman’s Box2D lectures. Once she gets this down, she’s adding collision detection; the BEHEMOTH should be able to kick the stars, stomp on them, eat them back up, and generally be able to react to the starsoundvomit (that is the technical term).

In short :: it runs nicer now, we have sound vomit, & the general springiness/animation of the puppet is getting a bit of an overhaul.

Is there any particular feature you REALLY want to see on the BEHEMOTH, other than what we have? We’re focusing on just polishing the one puppet, rather than rushing to make three puppets that are just ehhhhh.

So after several miserable knitting machine swatch failures, I’m come to the conclusion that in order to create a visually appealing pattern, I’m going to have to work forwards and backwards, making sure that the generative pattern actually conforms to more “rules” of knitting rather than being purely chaotic.

I found an awesome book, “Creative Knitting: A New Art Form” by Mary Walker Phillip which is chock full of stuff like this which really made me want to step up my aesthetic game.

I also need more stitch variation than the previous sketches were producing in order to create something like this. At the very least eyelets and cables which need to run in parallel lines. So taking a suggesting from Golan I started playing with reaction diffusion that can be spawned off of underlying layers of different equations in order to create more variety in stitch pattern and also more interesting forms. I still like the idea of using it to “grow” your own skin, since the same algorithm is used to produce so many different patterns in nature (often for pelts that we kill for and make into clothing anyway). I’m aiming for a user interactive app that takes some input into the system and generates the pattern for a swatch as far as the software end goes

So here are some initial attempts in processing…

But really its that time again where Processing reaches its limit and its time to port over to Cinder. Fortunately Robert Hodgin and rdex-fluxus had some sample Shader scripts online specifically for reaction diffusion so it was much easier to get started that I initially feared.

Andrew from the Museum of Arts and Design got back to me as well. Unfortunately his machine doesn’t do cables, but I could try substituting different kinds of stitches to mimic the effect.

In the meantime, I bought a super simple SingerLK off of craigslist, and after a pretty rocky learning curve, its actually pretty simple to generate alot of fabric quickly by hand. Also, you can pretty much do any kind of stitch on it (if you’re willing to do the extra work to set it up. So since I’m trying to suppress the gnawing fear that the whole send out my pattern and get it nicely printed and sent back in time for the show is going to blow up in my face, its reassuring to know that if I had to, I’m pretty sure I could generate at least swatches myself.

Some additional cheap tasks

I collected ~600 statements I collected about robots in the future (see previous post). Now I asked people to judge these statements as follows:

• This is good for our future {agree/disagree in a 7-point scale}
• The statement makes me feel … {anger, disgust, fear, happiness, sadness, surprise}
• I foresee a bad future if the statement becomes true {agree/disagree in a 7-point scale}

This was fast! I collected 5 responses from different workers for each set of answers (paying $0.01 per HIT).

And let people draw!

I asked people to hand draw a robot. The robot had to be designed according to one of the statements and, once drawn, the workers had to upload a picture of it to MTurk. The next picture shows the type of responses I got when paying $0.05 per drawing:

Instead of asking people to take a picture, I set up a website using Google App Engine to let them draw this time. The good side of letting people draw with the mouse is that they are obligated to make a composition, and that each drawing is going to be unique at the end. The bad side to me is the more mouse-style kind of drawing that results…

I made the drawing application using processing.js:

The tricky pars consists in converting what people draw with the processing script into an image, and pushing that into the server. The way I did it was using HTML5 and jQuery:

<div style="float: left; width: 600px">
<!-- This is the canvas where people draw, managed by processing.js -->
<canvas data-processing-sources="/js/processing/draw-robot.pjs"></canvas>
</div>
 
<!-- This is the form that allows to submit the drawing -->
<div style="float: left;">
<form id="canvasForm" method="post" action="/draw">
<input type="hidden" name="idea" id="idea" value="{{ idea }}"/>
<div><label>Name your robot:</label></div>
<div><textarea name="robotname" id="robotname" rows="1" cols="40"></textarea></div>
<div>
<p>
Click the 'Save Drawing' button after you have drawn the robot.<br/>
Make sure it looks as required.
</p>
</div>
<div><input type="submit" value="Save Drawing" name="canvasFormSaveButton"/></div>
</form>
 
<div id="result"></div>
</div>
 
<!-- When the input button of the form is pressed, we prevent normal execution -->
<!-- convert the canvas to an image using the function toDataURL() and, finally, -->
<!-- submit the POST request -->
<script type="text/javascript">
 
$("#canvasForm").submit(function(event) {
 
event.preventDefault(); 
 
$("#result").html( "Processing... Don't reload or close the page." );
 
var $form = $( this );
var ideaval = $form.find( 'input[name="idea"]' ).val();
var robotnameval = $form.find( 'input[name="robotname"]' ).val();
 
$.post( "/draw", 
{ 
  idea: ideaval,
  robotname: robotnameval,
  format: "png",
  img: document.getElementsByTagName("canvas")[0].toDataURL()
},
      function( data ) {
          var content = $( data );
          $("#result").html( content );
      }
    );
});
</script>

This script doesn’t work pretty well with Internet Explorer, so also I redirect the user to an error page if IE is detected:

<script>
  if ($.browser.msie){
  window.location.replace("/draw");
  }
</script>

The backend of the app runs in python:

class DrawRobot(webapp.RequestHandler):
    dataUrlPattern = re.compile('data:image/(png|jpeg);base64,(.*)$')
 
    def post(self):
        robot = RobotPic()
        robot.idea = int(self.request.get("idea"))
        robot.name = self.request.get("robotname")
        robot.format = self.request.get("format")
        img = self.request.get("img")
 
        imgb64 = self.dataUrlPattern.match(img).group(2)
        if imgb64 is not None and len(imgb64) > 0:
            robot.image = db.Blob(base64.b64decode(imgb64))
            robot.put()
            robotid = robot.key().id();
            self.response.out.write("""
<div id="content">
<p>
Thanks. Your drawing was saved.<br/><u>Your identifier for MTurk is  
<span style="color:#FF0033">%s</span>.</u>
</p>
</div>""" % (robotid.__str__()));
 
        else:
            self.response.out.write("<div id='content'>Nope. Didn't work!</div>");

Here are some of the drawings I got by paying $0.03:

CNC MERINGUE – FAIL

Mixing meringue.

Attempt at pushing and pulling meringue peaks.

Various tooling attempts, some 3d printed, some household utensils.

Unfortunate results…

CNC Meringue from eric brockmeyer on Vimeo.

.
.
.
.
.
.
.
.
.
.
CNC M AND M’S

M and M pattern.

M and M setup including, CNC machine, hacked vacuum cleaner, and LinuxCNC Controller.

Pick and place M and M machine.

Results.

CNC M and M’s from eric brockmeyer on Vimeo.

Here is a brief update with Interactive Robotic Fabrication…these are a few screen shots of the RAPID code running in Robot Studio which is receiving quaternion coordinates from openFrameworks via an open socket connection over wifi. Currently the openNI library is reading the skeleton / gesture or the right arm and producing a vector at the continuation of the neck to arm. This gives the robot an orientation that is relative to the users position.

Setup Begins with openNI and ofx which produces the RAPID code to send to Robot Studio…

making the socket connection…

specific IP address. This may interesting if the user can be anywhere within the same wifi network and controlling a robot someplace else…

Once the socket connection has been made the port for receiving the coordinates begins to listen…This is a view of the Virtual Controller which mimics the physical one.

Last is a short video of actual live control of the robot simulation via the process described above.

Plan for interaction….
I have ordered 50lbs of clay and lots of foam to expirement with the kneeding / poking of clay. The tool that will be used is like a pyramid which will yield a pattern something like Aranda Lasch furniture.

The good news is that I’m still obsessed, enamored, and re-watching the same inspiration animation over and over. The bad news is, everything (else) has been seriously overwhelming and I haven’t been able to reciprocate the love. Not going to lie about it.

Just thinking/researching about the pieces which will make this work, keeping in mind that half the battle is learning the algorithm and learning to code better. Will also try to leverage a strength to make it come together: visuals.

[ ] Optical Flow

Optical Flow.

w/ Distortion.

[ ] Edge Detection

Edges in action.
Reading: Other Methods of Edge Detection, Edge Detection via Processing

[ ] Sepia Colorization
TBD

Robots and the future?

I would like to collect opinions about robots in the future (what they will be doing and what they should do). I decided to try Amazon Mechanical Turk, and started paying little cents for people to give ideas about robots.

I had to use different strategies to get responses from people. I started with a very descriptive task with more than 20 lines of descriptions and examples about how to complete it. The task consisted in completing 6 sentences: “In X years, some robots will …”; and “In X years, robots should …” with X = {10,20,30}. Not many people filled the survey, and so I had the impression that too much description at the beginning of a task might make people think it’s too hard and not worth the effort. I simplified my description over time, and changed how much I was paying many times. At some point seemed like paying too much was also counterproductive, as people started thinking that they had to do more than simply filling the short sentences. After several trials, I concluded that paying $0.03 for the 6 sentences was the more effective strategy.

There is always the possibility that people will cheat in MTurk. To try to reduce the amount of rejected tasks I was getting with my 3 cents, I decided to change the task to rather write two 30-words paragraphs about about what robots will and should do. I started paying 6 cents for the paragraphs without much luck; not many people was completing the task. I raised the incentive to $0.11, but just a few people reacted to the new price.

My last test consisted in asking about “robotics” instead of “robots”, as suggested by a few people in the class. During the past data collection, I realized that too many people was thinking about robots killing us, controlling us, etc.. and wanted to see if I could get some more diverse set of answers. I got just a few responses this time as well.

The unprocessed list of valid ideas I got about robots is presented at the end of this post.  Probably I ended up getting ~600 different ideas, though I still need to count the repetitions. I would like to be able to visualize the data in different formats (year, will/should, topic, agreement), so I started clustering the ideas by topic or by a single word description. Also, as soon as I organize a little better the data, I want to ask people to judge the ideas. Meaning, say how much you agree that robots will be doing this or that robots should be doing this other thing.

I would like to collect drawings about robots capable of doing what the people described. For this task I plan to ask people to draw by hand, take a picture, and upload it to MTurk. I expect price calibration to be an issue again :S

Finally, here’s the list of ideas (each sentence either starts with “In X years, some robots will…” or “In X years, robots should…”, but this is omitted to simplify the reading of the post),

Currently the RAPID output is working smoothly. The only issues currently are some coordinate system misalignment. The x,y,z orientations of the robot (RAPID), kinect (ofx), and the TCP (tool center point) are all different. We have solved most of these issues but just need to scale the input. Currently the interaction method is you move your hand and a vector is created from your chest to hand. That vector determines where the robot looks or where it operates.

Some hurdles we have covered….
+ Making openNI actually 3d instead of a flat image
+ RAPID output is clean with no errors (except joint limits when it moves to a position out of range)
+ Simulation is running live in Robot Studio (virtual controller) via OF

Things to do…
+ Map/Scale Quaternion inputs from openNI to limit input. This will minimize joint limit errors
+ Create a tool to operate on a soft material
+ Create a safe interaction

I’m revisiting the Kinect for my final project. I’m separating the background of an image from the foreground and using OpenGL GLSL Multitexturing Shaders to apply effects to the foreground.

GLSL Shaders work in OpenFrameworks, which is cool. However, there’s a trick that took me about three days to find. By default, ofTextures use an OpenGL plugin that allows for non-power of two textures. Even if you use a power of two texture the plugin is enabled and allocates textures that can’t be referenced from GLSL. FML.

The first GLSL shader I wrote distorted the foreground texture layer on the Y axis using a sine wave to adjust the image fragments that are mapped onto a textured quad.

I wrote another shader that blurs the foreground texture using textel averaging. You can see that the background is unaffected by the filter!

So there actually has been some great strides. I got in touch with Andrew Salomone through Becky Stern and he agreed to help me create a prototype on his computerized knitting machine sometime next week! Which definitely constitutes conquering probably the hardest “hard part”. He also sent me the manual to his machine so I can convert my patterns over into something recognizable.

Until then, I received this wonderfully bulky old LK100 Singer and maybe a pound of terrible acrylic off of craigslist with which to play around with. (Since I probably won’t have an excessive amount of time for prototyping on Andrew’s machine, might as well make sure the pattern looks right).  The lady who sold it to me showed me how to make a simple gauge swatch like so…..and also how to cable. Which is time consuming but braindead simple.

After looking at knitwear designers like Sandra Backlund, and the Cables after Whiskey sweater pattern  (that selects where cables will be placed with a probability function that produces pretty interesting results) both shown below, I want to create patterns with pseudo random cables/pleats in order to generate interesting volumes (ideally still very simple tube-based dresses).

As far as a “meaningful function”, perlin nosie/ flocking would  be sort of perfect, its so graceful, but I feel like that horse is quite dead by now. So I was looking at Reaction Diffusion algorithms, Grey Scott in particular. Given a standard boring ugly sweater pattern like this one, you can control the rate of diffusion to literally “grow” the diffusion to fit the pieces you are creating.

This creates very interesting results, especially if you knit so that the cables are nested within each other or maybe mapping each isolated circle to an eyelit. Since this pattern is actually impossible to translate into knitted cables due to their orientation and proximity, it could warp the fabric in good or bad ways. I’m also debating the merits of using an actual generative system, or just having at with simple probability.

This sketch begins with parallel lines, and then decides if a given line will intersect with any other with a random probability. I also realized this sort of technique could easily be applied to a sensor and collect some kind of data and visualize it that way. So I guess solving my previous “hard part” really has just given me alot more room to brood over this one. I think its important that its not just a textured knit produced electronically, but that the driving pattern underneath has some sort of relevance/significance other than pure chaos.

I’m working on reproducing the Blob at http://hakim.se/experiments/html5/blob/03/. So far, I’ve gotten box2d on javascript working, and am working on trying to implement a blob with the same kind of fluidity and dynamic.
I’ve thought about implementing a bunch of particles attached by springs, where each particle is connected to by an even number of springs, half on the left and half on the right.