Author: shuann

shuann-arsculpture

 

I created this playful AR experience specifically for the night light in my bedroom to add more liveliness to the moment. The materials and particles used in the scene are intentionally made glowy, thus contrasting the surrounding environment and can be viewed as part of a virtual night light. The fishes swim around the viewer in circular motions with different velocities and distance.

Given the way of how the found model was made, the animation was first done with deformer which gives a better result: (this gif doesn't loop for some reason, so if you want to look at it please refresh the page)

However, unity does not support deformer animation import thus I had to go with rigging even though it fragments the mesh:

shuann-book

URL to ZIP: https://drive.google.com/file/d/1RQSg3IoQqV_9MWc9p3Vng2feqgH-IsUT/view?usp=sharing

Title: A Guide to Absurd Movies

One sentence description: Explore absolutely absurd movies with this chapter. Yet remember the biggest challenge is where to find them.

I first had the idea of making a recipe book for creating new movies. For the ingredients, I would just throw in some random director, actors, writers ect., and then you will generate a random plot to go along with it. However, as I discovered some resources online containing real user reviews I wanted to include that as well. Thus, eventually, my chapter become something like a review book on non-existing movies: it is the interplay between the fake and the real that I am personally very interested in.

The entire project is generated based on Markov chains because I have to have completely bazar plot lines which the computer think marks as possible with little intervention as possible (i.e. predefined grammar). However, also due to this technique that I am using, I had to spend hours and hours cleaning up the data that I was feeding into the program. For example, I had to change all the names of the male protagonists to Chris and those of the female protagonists to Jade just so that names that only appear in one movie would not break the Markov chain. I also had to do a lot of filtering so that the plot content is not too over the place that two generated sentences adjacent to each other make no sense.

you can see that here, too many names and proper nouns are breaking the reading experience.

A lot better with cleaned up data, though more needed to be done.

Also, to make sure the first sentence makes sense, I need to incorporate many validators so that, for example, the sentence will always contain a determiner, does not start with a conjunction, etc. Moreover, I also tried to make sure that the pronoun matches the sex of the person that was named. Ex. "him" is changed into "her" because the determiner is identified as female in the first sentence.

Screenshots of the book pages: 

 var metaData;
var plot;
var posReview, negReview;
var titleTracker = 1;
var dirList = [];
var writerList = [];
var actorList = [];
var plotList = [];
 
var linesPlot = [];
var linesReview = [];
var markovPlot, x = 160, y = 240;
var markovPosReview, markovNegReview, x1 = 160, y1 = 200;
 
var bannedBeginner = ["He", "His", "Her", "She", "They", "Their", "We", "Another", "Suddenly", "However", "The next"];
var bannedMid = ["then", "as well", "also", "not only"];
var male = ["he", "Chris", "Chris's", "him", "male", "man", "his", "gentlemen", "boy", "himself"];
var female = ["she", "Jade", "Jade's", "her", "female", "woman", "her", "lady", "girl", "herself"];
 
var finalReivew = [];
var curDir = [];
var curActor = [];
var curWriter = [];
 
var movies = [];
 
function preload(){
  metaData = loadJSON("movieDetails.json");
 
  plot = loadStrings('illusion.txt');
  posReview = loadStrings('posReview.txt');
  negReview = loadStrings('negReview.txt');
}
 
function setup() {
  createCanvas(600, 600);
  textFont('times', 16);
  textAlign(LEFT);
 
  // create a markov model w' n=4
  markovTitle = new RiMarkov(2);
  markovPlot = new RiMarkov(3);
  markovPosReview = new RiMarkov(4);
  markovNegReview = new RiMarkov(4);
 
  // load text into the model
  markovTitle.loadText(plot.join(' '));
  markovPlot.loadText(plot.join(' '));
  markovPosReview.loadText(posReview.join(' '));
  markovNegReview.loadText(negReview.join(' '));
 
  extractInfo();
 
  for (var i = 0; i < 15; i++){
    generate(i);
  }
 
  // Create a JSON Object, fill it with the Poems.
  var myJsonObject = {};
  myJsonObject.movies = movies;
  // console.log(myJsonObject);
 
  // Make a button. When you press it, it will save the JSON.
  createButton('SAVE POEMS BUTTON')
    .position(10, 10)
    .mousePressed(function() {
      saveJSON(myJsonObject, 'myPoems.json');
    });
}
 
function combineText(index) {
  var finalPlot = linesPlot.join(' ');
  for (var i = 0; i < linesReview.length; i++){
    finalReivew[i] = linesReview[i].join(' ');
  }
 
  var title = "Absurd Moive No." + titleTracker;
  titleTracker += 1;
 
  movies[index] = new moiveInfo(title, curDir, curActor, curWriter, finalPlot, finalReivew);
  console.log(movies[index]);
}
 
function generate(index) {
  randBasicInfo();
 
  linesPlot = markovPlot.generateSentences(2);
  checkPlot();
  checkAgreement();
 
  for (var i = 0; i < 3; i++){
    if (random() < 0.7){
      linesReview[i] = markovNegReview.generateSentences(random([3, 4]));
    } else {
      linesReview[i] = markovPosReview.generateSentences(random([2, 3, 4]));
    }
  }
 
  combineText(index);
}
 
//generate basic movie setp info
function randBasicInfo(){
  curDir = [];
  curActor = [];
  curWriter = [];
  var dirNum = 1;
  var actNum = random([3, 4, 4]);
  var wriNum = random([1, 2, 2]);
 
  for (var a = 0; a < dirNum; a++){
    curDir[a] = random(dirList);
  }
 
  for (var b = 0; b < actNum; b++){
    curActor[b] = random(actorList);
  }
 
  for (var c = 0; c < wriNum; c++){
    curWriter[c] = random(writerList);
  }
}
 
function checkPlot() {
  //get all words in the frist sentence
  var sentence1 = linesPlot[0].split(' ');
 
  //replace first sentence when it begins with pronoun
  if (checker(sentence1[0], bannedBeginner) || checkConjunction(sentence1[0])){
 
    linesPlot[0] = markovPlot.generateSentence();
    checkPlot();
  } else {
    //look at each word in the sentence
    for (var i = 0; i < sentence1.length; i++){
      sentence1[i] = RiTa.trimPunctuation(sentence1[i]);
    }
    // console.log(sentence1);
    // console.log("here");
 
    // if (checker(sentence1, bannedMid) && checker(sentence1, charNames) === false){
    //   console.log(linesPlot[0]);
    //   linesPlot[0] = markovPlot.generateSentence();
    //   console.log(linesPlot[0]);
    // }
 
    if (checker(sentence1, bannedMid)){
      linesPlot[0] = markovPlot.generateSentence();
      checkPlot();
    }
 
    if (checkDeterminer(sentence1) === false){
      linesPlot[0] = markovPlot.generateSentence();
      checkPlot();
    }
  }
}
 
// https://stackoverflow.com/questions/37428338/check-if-a-string-contains-any-element-of-an-array-in-javascript
function checker(value, ban) {
  for (var i = 0; i < ban.length; i++) { if (value.indexOf(ban[i]) > -1) {
      return true;
    }
  }
  return false;
}
 
function checkDeterminer(value){
  for (var a = 0; a < value.length; a++){
    if (RiTa.getPosTags(value[a]) == "dt"){
      return true;
    }
  }
  return false;
}
 
function checkConjunction(value){
  var tag = RiTa.getPosTags(value);
  for (var i = 0; i < tag.length; i++){
    if (tag[i] === "in" || tag[i] === "cc"){
      return true;
    }
  }
  return false;
}
 
function checkAgreement() {
  var sex = null;
  // sentence1Agg();
  // sentence2Agg();
 
  var sentence1 = linesPlot[0].split(' ');
  var sentence2 = linesPlot[1].split(' ');
 
  for (var i = 0; i < sentence1.length; i++){ sentence1[i] = RiTa.trimPunctuation(sentence1[i]); if (sex === null){ if (male.indexOf(sentence1[i]) > -1){
        sex = "M";
      }
      if (female.indexOf(sentence1[i]) > -1){
        sex = "F";
      }
    }
  }
 
  for (var a = 0; a < sentence1.length; a++){ if (RiTa.getPosTags(sentence1[a]) == "prp"){ if (male.indexOf(sentence1[a]) > -1 && sex === "F"){
        var index = male.indexOf(sentence1[a]);
        sentence1[a] = female[index];
      }
 
      if (female.indexOf(sentence1[a]) > -1 && sex === "M"){
        var index = male.indexOf(sentence1[a]);
        sentence1[a] = male[index];
      }
    }
  }
 
  linesPlot[0] = sentence1.join(' ') + ".";
}
 
function moiveInfo(title, dir, actor, writer, plot, reviews){
  this.title = title;
  this.direc = dir;
  this.actor = actor;
  this.writer = writer;
  this.plot = plot;
  this.reviews = reviews;
}
 
function extractInfo(){
  console.log(metaData.results.length);
  for (var i = 0; i < metaData.results.length; i++){ if (metaData.results[i].imdb.rating >= 8){
      //extract the names of directors on file for all movies with imdb rating > 8
      if (metaData.results[i].director != null){
        if (metaData.results[i].director.length > 1){
          var dirArray = metaData.results[i].director.split(', ');
          for (var a = 0; a < dirArray.length; a++){ append(dirList, dirArray[a]); } } else { var dirArray = metaData.results[i].director; append(dirList, dirArray); } } if (metaData.results[i].writers != null){ if (metaData.results[i].writers.length >= 1){
          for (var b = 0; b < metaData.results[i].writers.length; b++){ append(writerList, metaData.results[i].writers[b]); } } } if (metaData.results[i].actors != null){ if (metaData.results[i].actors.length >= 1){
          for (var c = 0; c < metaData.results[i].actors.length; c++){
            append(actorList, metaData.results[i].actors[c]);
          }
        }
      }
    }
  }
}

Sources:

review.json from: https://github.com/Edmond-Wu/Movie-Reviews-Sentiments

Plot summary dataset from: http://www.cs.cmu.edu/~ark/personas/

Example movie dataset from: https://docs.mongodb.com/charts/master/tutorial/movie-details/prereqs-and-import-data/

shuann-parrish

Above all, I think the way she defines her text robots as explores that explore "whatever parts of language that people usually find inhospitable" are very inspiring to me. Personally I have never though of generative texts in this way. I agree that we often want to create robots that can closely assemble humans, not matter if it's a bot that plays chess or reconstructs languages. Thus, embracing the rawness and the awkwardness of the content that was created by a machine and more importantly, find meanings within it can in some sense open up possibilities to new ways of understanding languages, and more specifically, the holes that we had naturally avoided.

Another that stuck with me is the phrase that "gaps could have been there for a reason", as it might have indications of violence or other things that are harmful. I think this is an important point to make. When we make automated machines and let them out in the world, we often consider what they create are out of our control, and it is just what it is (ex. TayTweets). However, I totally agree with the speaker that we, as the creator of the bots, need to take on the responsibility to make sure that we are actively take percussions against those undesirable outputs.

shuann-LookingOutwards04

I think the idea of a robot that gradually builds itself is an extremely interesting! Not only was the fiberbot able to construct itself over time, it was also creating very organic shapes that one would probably find in nature, despite the fact that itself was a pertly mechanic object. The bot is placed in to the nature (on a outdoor grass field, completely exposed to the outside environment) instead of being displayed in a museum/lab setting which I see as an artistic choice. It defiantly adds to the conversation happening between the artificial an the natural, which is what really caught my eyes. Moreover, I think this piece really stands out from the other ones that I have looked at so far because it can be almost seen as a performance work that last over a few days. It is not only the final structural form that was the art work but more so the slow process of continuous self-construction in the field.

Link to project: https://www.creativeapplications.net/robotics/fiberbots-framework-for-cooperative-swarm-robotic-manufacturing/

shuann-Body

Link to app: https://detailed-feet.glitch.me/

GIF:

Sketch:

My original idea was pretty different from what I eventually finished. Originally I wanted to create a drawing tool that one can draw with his/her pupils. The line would trace where the eye had been and the entire canvas would erase itself if the person blinks. However, I was having trouble to distinguish different state of the eyes with the face tracking templet code and decided to go to a different direction.

Then the new idea I had was to use the cereal head movement to control a basic character. I first thought of the more conventional multiple level of floors that a character could be placed into by change the vertical position of the head. However I realized that did not make much sense since I want to use head turn to trigger "jump". The left&right motion does not naturally map to the up&down motion of the jump. Therefore I redesigned the floor to be vertical and have the character run up the screen to avoid obstacles. That been said, this not a game yet as it does not have any level design, no goals to achieve, etc. As it for now, it's more like an exploration on the kind of controllers that can be implemented. If there is more time, I would like to turn this into a game that two people can play against each other at the same time.

Still Image:

 Video:

var local;
var obs = [];
var dir = [1, -1];
var player1 = null;
var stuck = false;
var score1 = 0;
var vel = 8;
 
var from;
var to;
var bgcX = 0;
 
function setup() {
 
  createCanvas(640, 480);
  background(0);
 
  //this is where posenet initlizes and webcam stuff happens
  PoseZero.init();
 
  //this is you, check out agent.js for adding properties and functions
  local = new Agent();
 
  player1 = new player(width/2,height*2/3,random(dir));
  obs[0] = new obstacle(width/2, height/2, random(15,21), random(dir));
  obs[1] = new obstacle(width/2, height/3, random(15,21), random(dir));
 
}
 
function draw() {
  background(255);
 
	local.update(PoseZero.get());// update your skeleton
 
  push();
  if (local.data.pose != null){ 
    push();
    if (player1.dir === 1){
      bgcX = width/2;
      from = color(255,100,147);
      to = color(255,182,193);
    }
    if (player1.dir === -1){
      bgcX = 0;
      from = color(255,182,193);
      to = color(255,100,147);
    }
    noStroke();
    fill(150);
    var widthRect = width/40;
    noStroke();
    for (var i=0;i<20;i++){
      inter = lerpColor(from,to,(i+1)*(1/20));
      fill(inter);
      rect(bgcX + i*widthRect,0,widthRect,height);
    }
    pop();
 
    PoseZero.draw_pose(local.data.pose,{color:local.data.color})
 
    if (player1.dir === 1){
      text("shake your head RIGHT" , width/2 + 170, height - 20);
    }
    if (player1.dir === -1){
      text("shake your head LEFT ", 20, height - 20);
    }
 
    push();
    drawPlayground();
 
    palceObs();
    boundaryCheck();
    speedCheck();
 
    if (player1.stuck() === false){
      for (var elem = 0; elem < obs.length; elem ++){
        obs[elem].draw();
        obs[elem].move();
      }
    } else {
      for (var elem = 0; elem < obs.length; elem ++){ obs[elem].draw(); } } player1.draw(); player1.checkDir(); player1.headTurn(); pop(); } pop(); } function palceObs(){ if (frameCount % 15 == 0){ if (obs[obs.length - 1].y != -20){ obs.push(new obstacle(width/2, -20, random(15,21), random(dir))); } } } function boundaryCheck(){ if (obs[0].y >= height){
    obs.shift();
  }  
}
 
function speedCheck(){
  var closeness = PoseZero.estimate_scale(local.data.pose);
 
  if (closeness <= 70 && closeness > 40) {
    vel = 8;
  }
 
  if (closeness <= 100 && closeness > 70) {
    vel = 11;
  }
 
  if (closeness <= 120 && closeness > 100) {
    vel = 14;
  }
}
 
function drawPlayground(){
  var road = width/2;
  strokeWeight(4);
  line(road, 0, road, height);
 
}
 
function obstacle(x,y,s,dir){
  this.x = x;
  this.y = y;
  this.dir = dir;
  this.s = s;
  this.r = int(random(240,255));
	this.g = int(random(30,200));
	this.b = int(random(50,69));
 
  this.draw = function(){
    push();
    stroke(0);
		strokeWeight(4);
		fill(this.r,this.g,this.b);
    if (this.dir == 1){
      rect(this.x, this.y, this.s * 4, 20, 30);
      for (var a=0; a<4; a++){
        push();
        stroke(250);
			  line(this.x + 6 + 15*a, this.y + 3, this.x + 15*a + 15, this.y + 17);
        pop();
		  }
    } else {
      rect(this.x - this.s * 4, this.y, this.s * 4, 20, 30);
      for (var b=0; b<4; b++){
        push();
        stroke(250);
			  line(this.x + 6 + 15*b - this.s * 4, this.y + 3, this.x + 15*b + 15 - this.s * 4, this.y + 17);
        pop();
		  }
    }
    pop();
  }
 
  this.move = function(){
    this.y += vel;
  } 
}
 
function player(x,y,dir){
  this.x = x;
  this.y = y;
  this.dir = dir;
  this.h = 50;
  this.w = 30;
  this.leg = 15;
  this.vel = 10;
  var triX;
  var jumpR = false;
  var toppedR = false;
  var jumpL = false;
  var toppedL = false;
 
  this.draw = function(){
    // console.log(this.dir);
    if (this.dir == -1) {
      triX = this.x - this.leg;
    } else {
      triX = this.x + this.leg;
    }
    push();
    strokeJoin(ROUND);
    strokeWeight(4);
    stroke(255);
    fill(255, 209, 124);
    line(triX + this.h*3/4 * this.dir, this.y + this.w/2, this.x + this.dir * 5, this.y + this.w/2);
    triangle(triX, this.y, triX + this.h * this.dir, this.y + this.w/2, triX, this.y + this.w);
    ellipse(triX + this.h*3/4 * this.dir, this.y + this.w/2, this.leg*2 , this.leg*2);
    stroke(0);
    line(triX + (this.h*7/8 - 4) * this.dir, this.y + this.w/2 - 6, triX + this.h*6/8 * this.dir, this.y + this.w/2 - 6);
    line(triX + (this.h*7/8 - 4) * this.dir, this.y + this.w/2 + 6, triX + this.h*6/8 * this.dir, this.y + this.w/2 + 6);
    pop();
  }
 
  this.checkDir = function(){
    var playerPos = PoseZero.nosePosX(local.data.pose);
 
    if (playerPos < width/2) {
      this.dir = -1;
    } else {
      this.dir = 1;
    }
  }
 
  this.headTurn = function(){
    //check for jumps
    if (PoseZero.rightTurn(local.data.pose) && this.dir == 1){
      jumpR = true;
    }
    if (PoseZero.leftTurn(local.data.pose) && this.dir == -1){
      jumpL = true;
    }
 
    if (jumpR === true){
      this.jumpRight();
    } 
    if (jumpL === true){
      this.jumpLeft();
    }
  }
 
  //Right jump animation
  this.jumpRight = function(){
    if (toppedR === false && this.x < width/2 + 100){ this.x += this.vel; if (this.x >= width/2 + 100) {
        toppedR = true;
      }
    } 
    if (toppedR === true && this.x != width/2){
      this.x -= this.vel;
      if (this.x <= width/2) { this.x = width/2; jumpR = false; toppedR = false; } } } //Left jump animation this.jumpLeft = function(){ if (toppedL === false && this.x > width/2 - 100){
      this.x -= this.vel;
      if (this.x <= width/2 - 100) { toppedL = true; } } if (toppedL === true && this.x != width/2){ this.x += this.vel; if (this.x >= width/2) {
        this.x = width/2;
        jumpL = false;
        toppedL = false;
      }
    }
  }
 
  this.stuck = function(){
  //check for obstacles
    for (var elem = 0; elem < obs.length; elem ++){
      var obsTop = obs[elem].y;
      var obsBottom = obs[elem].y + 20;
      var obsOuter = obs[elem].x + obs[elem].s * obs[elem].dir * 4;
      var obsInner = obs[elem].x;
      if (obs[elem].dir == this.dir && obsTop < this.y && obsBottom >= this.y){
        if (this.dir === 1 && this.x <= obsOuter){ return true; } if (this.dir === -1 && this.x >= obsOuter){
          return true;
        }
      }
    }
    return false;
  }
}

 

shuann-LookingOutwards03

The project I selected is a kinematic maze called Landscape Abbreviated created by Nova Jiang. I think the interaction here is extremely interesting because, unlike many other interactive work, the initiative is held by the art work itself. The installation purposefully fills the room so that the visitor has to walk through the maze before going to the next room. Yet which specific path they will eventually end up taking is determined by the installation itself. The modular elements that consist the work can individually turn at different angles and different time thus forcing the visitors to his/her unique path. The simple yet effective unpredictability of this piece makes it an extremely intriguing work to interact with.

Link to work:https://www.novajiang.com/projects/landscape-abbreviated/