Final rendered video on Vimeo:

Screenshot of the work right before rendering:

Screenshot of the scripting device in Maya:

Gif of time slider animation pre-render in Maya (you may need to click it to see it run):

Sketches I did of the characters:

For my project, Golan suggested that instead of using Processing or Three.js, I could learn scripting in Maya because of my interest in animation. I was very excited to start this project, and took to it with a more story-focused mindset with the motion capture than I think most of the class did. I wanted to use the scripting to do things in Maya that I couldn’t do by hand (or at least couldn’t bear to do by hand or in the given time frame) that would supplement a story, no matter how short. The initial idea I had for this was  a pair of disgraced/fallen/unfit samurai that circled each other in blame, getting closer and farther together with an audience of masks turning to look always at the two of them and closing in gradually. Eventually, I realized I didn’t have time to model two samurai and settled on modelling the shell (mask, gloves, socks, cape) of a disgraced/fallen/unfit samurai warrior and trying to achieve a feeling of melancholy and nostalgia for a better time. I wanted to use python scripting to generate and randomly place another modelled mask, and make it so that whenever the main mocap samurai moved, the masks would turn their faces to always follow him. Starting this project, I watched video tutorials on how to python script in Maya, following along with them. After figuring out if I could do what I wanted to do, which actually the video tutorials basically covered, I started modelling. Before this project, I had only had a bit of basic modelling experience and a general broad overview of what Maya could do. The modelling ended up taking me more time than I thought. Afterwards, I also learned how to import a BVH file into Maya and how to rig/bind a model to the BVH skeleton. When I got to coding, I came to an unexpected circumstance. Although the masks would turn to face the samurai, after the samurai was binded to the skeleton, this no longer worked. At first I tried to bind the skeleton different ways, but in the end I made a separate object that I gave 100% transparency that I hand animated to follow the samurai around. The masks then followed that object. In the end, I didn’t end up liking the effect of the turning masks because they made the scene more confusing because the masks didn’t turn enough to be horribly noticeable. After finally getting everything set up and moving, I learned how to render. This is the first time I’ve rendered a scene, and I didn’t expect the end number of frames to be around 2000. The 2000 frames took longer to render than I thought they would. I tried to change the frame rate to 24 fps, but doing so significantly slowed down the mocap. The final step was to take my rendered scenes and stitch them together in Premiere. The end product was slower than it looked in Maya so I sped it up, utlimately shortening it by half, and also rendered darker than my test frame renders. I didn’t have time to re-render all the frames, but I think it was good experience going into the next time I try to render something. In the end I think I’m satisfied with the project, but I would definitely like to do more with it given more time to really get things to move, thinking more interactively along with my story-focus and getting more interactivity (leaving enough time for when things I want to work out don’t and so on). I want to utilize code more and dig deeper into what I can do with it, and also learn more the Maya-Python vocabulary.

Once again the WP-Syntax tool still hates me, and so here is the Github link to the code:

Code

Mocap is cool. This project was fun just to get my hands on 3D software and also to actually see a mocap setup for the first time. Being my own model was not so great (my ‘performance’ is not very compelling, though I did try to do one of the dances from my generative book – just the foot pattern without much flourishing). Doing this reminds me I need to expand my network in Pittsburgh of performers, dancers, etc. – which I will do.

I didn’t write code for my final output, but I did get Golan’s example code working in Processing with my BVH. Then I moved onto exploring the 3D animation software, Cinema 4D. I’d learned a little of this program about two years ago, so it was great to get back into it a little. I think I’ll try more things with this software now. I know that scripting in Python is possible in Cinema 4D. I didn’t script in this project, but would try this on the second iteration.

The project was fun. My output isn’t thrilling, but I’m glad to play with 3D (and remember why I love editing animation/video) and learn about cloners, physics tags (rigid body, collider body, force, friction, etc), lighting effects, and using the mocap skeleton.

Here’s the video:

For my mocap project I wanted to do a study of the nCloth feature in Maya used with motion as well as get a basic grasp of the capabilities of scripting. In both aims I think I was quite successful. Each gif below is taken from separate playblasts (screencasts), all of which can be downloaded here--they chronicle the process of getting the result above.

Process

To start I knew I wanted some fairly clean mocap data– capturing it myself would come with its own set of challenges. Mixamo‘s animation library is pretty extensive and set up with Maya takes practically no time (setting up the auto-rig feature is simple, easy and most importantly free), so I set up a simple bellydancing animation and looked at the character’s skeleton. The first script (2nd picture on the left) was basically a test which iterated through the skeleton and parent an object at its x and y coordinates. If one does not want any joints in the chain to have an object parented to them (such as the fingers, which were not very crucial in this particular animation) its easy enough to unparent them Mixamo skeleton and place them in a separate group.

 

My second script essentially did the same as the first but for a polyPlane instead (pictured bottom left). These would become nCloth once the feature was applied.

 

The most time-intensive part of the project was experimenting with the nCloth feature, which I knew to be pretty finicky to work with; keeping cloth simulations from glitching and flying in unexpected directions takes time. Tutorials are any Maya-user’s best friend, so I found a quick but helpful tutorial using a transform constraint to keep the cloth moving with the dancing form. My third script produced the gifs shown below, which essentially put into action each step the tutorial instruction but in code form.

Finally, my last script loops third script to create the final product shown below (minus the shading material). I ran the first one to create and parent spheres at every joint except the fingers, then ran the second one to create a plane at each joint as well. The last script iterates through each of those spheres and planes and assigns a collider, nCloth, (respectively) and then applies a transform constraint to the two, so the cloth follows the parented spheres.  If one wishes to run the script more than once or on different objects, the iteration number must be updated accordingly, since when Maya creates nCloth it names it “polySurface” and then the next number in the outliner.

Conclusion

From this project, I learned that scripting isn’t that hard! Essentially all you are doing is translating into code every action you would be doing manually. Commands can easily be looked up, and even someone with limited knowledge of Python would be able to pick up on it quickly. There’s also a reference describing every command and its flags. One can even call the maya.mel.eval function which directly calls a MEL shell command. It made a project which would’ve been possible yet painstaking to do manually fairly quick and simple.

 

 

 

 

 

 

FINAL PRODUCT

 

SKETCHES

SCREENSHOT OF PROGRAM

SUMMARY

For this project, I really wanted to alter some characteristics  of previously created narratives, in hopes of changing their concepts. My initial idea consisted of imitating lead roles in films and switching their living forms with inanimate objects. (i.e. – Replace movie characters with the tools/objects they use.)

PROCESS

When coming up with possible movies to imitate, I regarded the key objects (i.e. – staff, gun, etc.) and how they related to their character’s role in the film (i.e. – police guard, wizard, etc.). The film that I thought would convey this the best was Quinton Tarantino’s Pulp Fiction. More specifically, I aimed to re-create Jules, played by Samuel L. Jackson, and a specific dialogue he has with one of his boss’s “business partners”. After reviewing the scene multiple times, I then decided to change up my concept and replace the main characters with a sort of visual pun (Hint: Pulp Fiction and Oranges).

After finalizing details, I recorded multiple BVH files of Jules and the business partner, Brett. This process was a bit difficult since the camera used (Kinect V2) didn’t particularly like the fast movements I was trying to imitate while standing and sitting. As a result, some of the movements came out a little glitchy and some of the previous “aggressive” movements had to be slowed down.

After recording, I inputted the BVH files and adjusted camera angles similar to those in the actual scene. This took quite a while, as timing was key. After the scenes were lined up, I proceeded to create a set that would fit the new concept I was aiming for (i.e. – kitchen counter). I then rendered out the figures and adjusted certain characteristics at certain points of the film. For example, when the Brett Orange is shot, his color begins to change to a greener, more vile color.

REVIEW

I am particularly happy with the results I created. Although the rendering of the characters is not as high of quality as I would like for it to be, I am happy with the results given a rather chaotic week.

I will definitely continue to make this project better in the future (i.e. – work on developing software to automatically rotor-scope an inputted scene, make adjustments to character rendering for smoother movement, etc.). Once I have a better understanding of the bugs I’m facing and also have created more efficient programs to render out these scenes, I may even continue to recreate the entire film!

 

GitHub Link: https://github.com/xapostol/60-212/tree/master/Deliverables%208

// Renders a BVH file with Processing v3.2.1
// Note: mouseX controls the camera.
import ddf.minim.*;

PBvh1 orngJ;
PBvh2 orngB;
PImage bg1; // background
PImage bg2; // background 2

// Time
int m;

AudioPlayer player;
Minim minim; // audio context


//------------------------------------------------
void setup() {
  size( 1280, 720, P3D );
  
  // Load a BVH file recorded with a Kinect v2, made in Brekel Pro Body v2.
  orngJ = new PBvh1( loadStrings( "jules_00.bvh" ) );
  orngB = new PBvh2( loadStrings( "brett_00.bvh" ) );
  
  // Load the soundfile
  minim = new Minim(this);
  player = minim.loadFile("Pulp Fiction - Jules and his Bible Verse1_01.mp3", 2048);
  player.play();
  bg1 = loadImage("background_02.jpg");
  bg2 = loadImage("background_01.jpg");
}

//------------------------------------------------
void draw() {
  background(bg1);
  m = millis();
  //println(m);   //Purposes of testing/timing for camera angles and effects.
  
  setMyCamera();        // Position the camera. See code below.
  //drawMyGround();     // Draw the ground. See code below. (Purposes Of Testing)
  updateAndDrawBody();  // Update and render the BVH file. See code below.
}


//------------------------------------------------
void updateAndDrawBody() {
  // Stop The Scene
  if (m > 118800) {
    m = 0; 
  }
  
  pushMatrix(); 
  translate(width/2+50, height/2, 10); // position the body in space
  scale(-1, -1, 1);                    // correct for the coordinate system orientation
  orngJ.update(m);                     // update the BVH playback
  orngJ.drawBones();                   // a different way to draw the BVH file
  popMatrix();
  
  pushMatrix();
  translate(width/2, height/2, -250);
  scale(-1, -1, -1);
  orngB.update(m);
  orngB.drawBones();
  popMatrix();
}


//------------------------------------------------
void setMyCamera() {
  // INITIAL CONDITIONS FOR CAMERA
    // Adjust the position of the camera
  float eyeX = width/2;            // x-coordinate for the eye
  float eyeY = height/3.0f - 500;  // y-coordinate for the eye
  float eyeZ = 500;                // z-coordinate for the eye
  float centerX = width/2.0f;      // x-coordinate for the center of the scene
  float centerY = height/2.0f;     // y-coordinate for the center of the scene
  float centerZ = -400;            // z-coordinate for the center of the scene
  float upX = 0;                   // usually 0.0, 1.0, or -1.0
  float upY = 1;                  // usually 0.0, 1.0, or -1.0
  float upZ = 0;                  // usually 0.0, 1.0, or -1.0


  //---------------------------------------------------------------------//
  //                          CAMERA ANGLES                              //
  
  background(bg1);
  // Angle #1 (Over Shoulder - BRETT)
  camera(eyeX-70, 0, -eyeZ, centerX, centerY, -1*centerZ, upX, upY, upZ);
  
  // Angle #2 (Over Top - JULES)
  if (m > 6600) {
    camera(width/2, height/3.0f - 250, 200, centerX, centerY, centerZ, upX, upY, upZ);
  }
  
  // Angle #1 (Over Shoulder - BRETT)
  if (m > 9500) {
    camera(eyeX-70, 0, -eyeZ, centerX, centerY, -1*centerZ, upX, upY, upZ); 
  }

  // Angle #3 (Wide)
  if (m > 10300) {
    background(bg2);
    camera(width/2, eyeY, eyeZ, centerX, centerY, centerZ, upX, upY, upZ);
  }
  
  // Angle #1 (Over Shoulder - BRETT)
  if (m > 17000) {
    background(bg1);
    camera(eyeX - 100, 0, -eyeZ + 200, centerX, centerY, -1*centerZ, upX, upY, upZ); 
  }
  
  // Angle #4 (Close Up - JULES)
  if (m > 24600) {
    camera(width/2 + 50, height/3.0f - 250, -60, centerX, centerY, centerZ, upX, upY, upZ);
  }
  
  // Angle #1 (Over Shoulder - BRETT)
  if (m > 31500) {
    camera(eyeX - 100, 0, -eyeZ + 200, centerX, centerY, -1*centerZ, upX, upY, upZ); 
  }
  
  // Angle #4 (Close Up - JULES)
  if (m > 36000) {
    camera(width/2 + 50, height/3.0f - 250, -60, centerX, centerY, centerZ, upX, upY, upZ);
  }
  
  // Angle #1 (Over Shoulder - BRETT)
  if (m > 44800) {
    camera(eyeX - 100, 0, -eyeZ + 200, centerX, centerY, -1*centerZ, upX, upY, upZ); 
  }
  
  // Angle #2 (Over Top - JULES)
  if (m > 48850) {
    camera(width/2, eyeY, 200, centerX, centerY, centerZ, upX, upY, upZ);
  }
  
  // Angle #4 (Close Up - JULES)
  if (m > 52000) {
    camera(width/2 + 50, height/3.0f - 250, -60, centerX, centerY, centerZ, upX, upY, upZ);
  }
  
  // Angle #1 (Over Shoulder - BRETT)
  if (m > 61000) {
    camera(eyeX - 100, 0, -eyeZ + 200, centerX, centerY, -1*centerZ, upX, upY, upZ); 
  }
  
  // Angle #4 (Close Up - JULES)
  if (m > 62000) {
    camera(width/2 + 50, height/3.0f - 250, -60, centerX, centerY, centerZ, upX, upY, upZ);
  }
  
  // Angle #4 (Close Up - JULES)
  if (m > 79000) {
    camera(width/2 + 50, height/3.0f - 250, -60, centerX, centerY, centerZ, upX, upY, upZ);
  }
  
  // Angle #1 (Over Shoulder - BRETT)
  if (m > 93000) {
     camera(eyeX - 100, 0, -eyeZ + 200, centerX, centerY, -1*centerZ, upX, upY, upZ); 
  }
  
  // Angle #5 (Tilt - JULES)
  if (m > 97000) {
    camera(width/2 + 50, height/3.0f - 300, -80, centerX, centerY, centerZ, -0.5, upY, upZ);
  }
  
  // Angle #1 (Over Shoulder - BRETT)
  if (m > 110000) {
     camera(eyeX - 100, 0, -eyeZ + 200, centerX, centerY, -1*centerZ, upX, upY, upZ);
  }
  
  // Angle #3 (Wide)
  if (m > 112800) {
    background(bg2);
    camera(width/2, height/6.0f - 1000, eyeZ, centerX, centerY, centerZ, upX, upY, upZ);
  }
  //---------------------------------------------------------------------//
}

void drawMyGround() {
  // Draw a grid in the center of the ground 
  pushMatrix(); 
  translate(width/2, height/2, 0); // position the body in space
  scale(-1, -1, 1);

  stroke(100);
  strokeWeight(1);
  float gridSize = 400; 
  int nGridDivisions = 10; 
  
  for (int col=0; col<=nGridDivisions; col++) {
    float x = map(col, 0, nGridDivisions, -gridSize, gridSize);
    line (x, 0, -gridSize, x, 0, gridSize);
  }
  for (int row=0; row<=nGridDivisions; row++) {
    float z = map(row, 0, nGridDivisions, -gridSize, gridSize); 
    line (-gridSize, 0, z, gridSize, 0, z);
  }

  popMatrix();
}