Here is my reimplementation of Text Rain by Camille Utterback and Romy Achituv. This is my first time working with Processing, and learning about object-oriented programming. I created a class called Letter that contains the object’s data and methods. Once I got the letters to fall, I then worked to connect the program to the webcam by importing a video library. The text rain program compares the brightness of the pixel of each letter to a threshold. If the brightness is more than the threshold, the letter rises up. If not, it falls down.

Here’s the code: https://github.com/pattvira/textRain

In my implementation of Textrain I read a string into an array of letter objects. Each object knows its character, xOffset, what column its in and where it is currently located vertically. Each object scans it’s column for the highest dark spot and then checks that against it’s current location. Thus, each object can make a determination about whether it should fall as normal or cling to the highest dark pixel.
A few details worth noting.

1. I flipped the pixel array so that the image is mirrored, this is nicer for display. 
2. I added a meager easing function to ‘captured’ letters to help keep them from wiggling
3. I basically ignore the bottom 50 pixels due to vignetting.

Textrain from john gruen on Vimeo.

Code is available at Github:https://github.com/johngruen/Textrain

import processing.video.*;
Capture v;
int totalOffset = 0; //helper var for counting xOffset for each Letter obj
PImage flip; //buffer for horizontally flipped image
String fallingLetters = "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Etiam non aliquam ante. Nullam at ligula mi. Nam orci metus";
Letter[] letters; 
PFont font;
int fontSize = 32;
float thresh = 100; // brightness threshold, Use UP and DOWN to alter
float sampleFloor = 40; //stop sampling pixels 20 rows from the bottom
color activeColor = color (255,0,255);
color fallColor = color(20,100);


void setup() {
  size(1280,720);
  v = new Capture(this,width,height,30);
  flip = createImage(width,height,RGB);
  font = loadFont("UniversLTStd-LightCn-32.vlw");
  textFont(font,fontSize); 
  initLetters();
}

void draw() {
  v.read();
  v.loadPixels();
  flip.loadPixels();
  flipImage(); //flip the image
  v.updatePixels();
  image(flip,0,0);
  for(int i = 0; i < letters.length; i++) {
    letters[i].draw();
  }
  flip.updatePixels();

}

void initLetters() {
  letters = new Letter[fallingLetters.length()];
  for(int i = 0; i < letters.length; i++) {
        letters[i] = new Letter(i,totalOffset,fallingLetters.charAt(i));
        totalOffset+= textWidth(fallingLetters.charAt(i));
  }
}

void flipImage() {
  for(int y = 0; y < v.height; y++) {
    for(int x = 0; x < v.width; x++) {
      int i = y*v.width + x;
      int j = y*v.width + v.width-1-x;
      flip.pixels[j] = v.pixels[i];
    }
  }
}

void keyPressed() {
   if (keyCode == UP) thresh++;
   else if (keyCode == DOWN) thresh--;
   println(int(thresh));
}


class Letter {
 int index; //just a good thing to know, used for debugging
 int xOffset; //offset horizontally of letter registration
 float speed; //speed
 char c; //what letter am i
 float curYPos, prevYPos; //current and previous y position. used for easing.
 int state = 0; // we use a tiny state switcher to control the flow. either the letter is falling or it isn't
 float topBrightPixel;//get top bright pixel;
 
Letter(int index_,int xOffset_,char c_) {
  index = index_;
  xOffset = xOffset_;
  c = c_;
  curYPos = int(random(-200,-100));//set currentYPos somewhere above the video
  speed = int(random(5,12));
} 

void draw() {
   senseBrightness();
   compareToCurrent();
   update();
   text(c,xOffset,curYPos); 
}

void senseBrightness() {
  topBrightPixel = 0;
  for(int i = xOffset; i < flip.pixels.length-flip.width*sampleFloor; i+=flip.width) {
    if(brightness(flip.pixels[i]) < thresh) { 
      break;
   }
   topBrightPixel++;
  }
}

void compareToCurrent() {
   if (topBrightPixel > curYPos + 2*speed || topBrightPixel >= flip.height - sampleFloor) {
     state = 0;
     //speed = random(5,12);
   } else {
     state = 1;
   }
}

void update() {      
    switch (state) {
      case 0:
        fill (fallColor);
        curYPos+=speed;
        speed = speed * 1.02;
        if (curYPos > height + 50)  {
          curYPos = random(-50,-200);
          speed = random(5,12);
        }
        break;
      case 1:
        fill(activeColor);
        curYPos += .6* (topBrightPixel - prevYPos);
        speed = random(5,12);
        break;
    }
      prevYPos = curYPos;
 }

}

I made this on my desktop mac mini, it doesn’t have a camera, so I used Kinect instead. The idea is the same. You can adjust the tilt using the arrow keys, and the threshold using “a” , “s” , “z” and “x” keys.

//TEXT RAIN KINECT//
//IACD ASSIGNMENT//
//CAN (JOHN) OZBAY//
//2013//

import org.openkinect.*;
import org.openkinect.processing.*;

Kinect kinect; //kinect settings
int kWidth = 640;
int kHeight = 480;
int kAngle = 0;

PImage depthImg; //depth settings
int minDepth = 60;
int maxDepth = 660;

String alphabet = "abcdefghijklmnopqrstuvqxyz";
String [] letters = new String [100];
float [] x = new float [100];
float [] y = new float [100];
boolean [] falling = new boolean[100];
int savedTime; //when the function was last called
int interval = 300; //how often we want to call the function(milis)
int bThreshold = 70;

void setup () {
smooth();
noStroke();
size(640, 480);
fill(0);
savedTime = millis();

size(kWidth, kHeight);
kinect = new Kinect(this);
kinect.start();
kinect.enableDepth(true);
kinect.tilt(kAngle);
depthImg = new PImage(kWidth, kHeight);
}

void draw () {
//depth threshold
int[] rawDepth = kinect.getRawDepth();
for (int i=0; i < kWidth*kHeight; i++) {
if (rawDepth[i] >= minDepth && rawDepth[i] <= maxDepth) { 
depthImg.pixels[i] = 0xFFFFFFFF;
} 
else { 
depthImg.pixels[i] = 0;
}
}
// draw threshold image
depthImg.updatePixels();
image(depthImg, 0, 0);
stroke(255,0,0);
line(10,434,160,434);
fill(255, 0, 0);
text("Tilt: " + kAngle, 10, 450);
text("Depth Thresh: [" + minDepth + ", " + maxDepth + "]", 10, 466);
for (int i=0; i savedTime+interval) { 
makeLetter(); 
savedTime = millis();
}
}

//name speaks for itself
void moveLetter (int tempI) {
y[tempI]++;
if (y[tempI] > height) { 
falling[tempI] = false;
}
}
//self explanatory 
void drawLetter (String tempS, float tempX, float tempY, int tempI) {
fill(0,200,255); 
text(tempS, tempX, tempY);
}

//Making of "The Letter"
void makeLetter () {
boolean madeLetter = false;
int randomNumber = int(random(alphabet.length()));
String tempChar = alphabet.substring(randomNumber, randomNumber+1);
for (int i=0; i 0 && loc <pixels.length) {
//get its brightness
float b = brightness(pixels[loc]);
if (b > bThreshold) {
isPixelDark = true;
}
}
return isPixelDark;
}

void keyPressed() {
if (key == CODED) {
//UP & DOWN ARROW KEYS, FOR KINECT TILT
if (keyCode == UP) {
kAngle++;
} 
else if (keyCode == DOWN) {
kAngle--;
}
kAngle = constrain(kAngle, 0, 30);
kinect.tilt(kAngle);
}

//A, S, Z, X - SET min Depth and max Depth
else if (key == 'a') {
minDepth = constrain(minDepth+20, 0, maxDepth);
} 
else if (key == 's') {
minDepth = constrain(minDepth-20, 0, maxDepth);
}

else if (key == 'z') {
maxDepth = constrain(maxDepth+20, minDepth, 2047);
} 
else if (key =='x') {
maxDepth = constrain(maxDepth-20, minDepth, 2047);
}
}

//don't forget to stop
void stop() {
kinect.quit();
super.stop();
}

Firstly, I use the camera to capture the users’ video. And use the code to capture people’s outline and fill it with black shadow.

Then, I write a randomly falling down letters generator. It produces randomly letters in the random position in the top. The letters will fall down if it doesn’t meet the black shadow.

Here’s code link:https://github.com/doukooo/textrain

import processing.video.*;

Capture video;
PImage show;
float threshold=50;
PFont f;

void setup(){
f=createFont("Arial",16,true);
size(320,240);
video=new Capture(this,width,height,50);
video.start();
}
int i=0,j=0;
int count=1;
FallLetter[] fallLet=new FallLetter[1];
FallLetter[] newFallLet;
void draw(){
if(video.available()){
video.read();
}
loadPixels();
video.loadPixels();
for(int x=0;x<video.width;x++)
{
for(int y=0;y<video.height;y++)
{
int loc=x+y*video.width;
color cur=video.pixels[loc];
float num=brightness(cur);
if(num>threshold)
{
pixels[loc]=color(255);
}
else
{
pixels[loc]=color(0);
}
}
}
updatePixels();
delay(50);
textFont(f,14);
fill(150,123,0);
int randI=int(random(640));
int randCh=int(random(65,90));
newFallLet=new FallLetter[count];
//deal with every letter
for(int n=0;n<count;n++)
{
if(n==count-1)
newFallLet[n]=new FallLetter(randI,randCh);
else
newFallLet[n]=fallLet[n];
char show=newFallLet[n].GetCh();
int xLoc=newFallLet[n].GetxLoc();
int yLoc=newFallLet[n].GetyLoc();
if(yLoc!=240)
{
text(show,xLoc,yLoc);
//if(red(pixels[xLoc+yLoc*width])==0&&green(pixels[xLoc+yLoc*width])==0&&blue(pixels[xLoc+yLoc*width])==0)
if(yLoc>0)
{
if(brightness(pixels[xLoc+yLoc*width])<100)
newFallLet[n].UpdateY(-1);
else
{
newFallLet[n].UpdateY(1);
}
}
}
}
fallLet=new FallLetter[count];
fallLet=newFallLet;
count++;

}
class FallLetter{

int xLocation,yLocation=10,ch;
char chShow;
FallLetter(int xLoc,int word)
{
xLocation=xLoc;
chShow=char(word);
}
//falling y position update
void UpdateY(int yLoc)
{
yLocation+=yLoc;
}
//get the position and char of the instance
int GetxLoc()
{
return xLocation;
}
int GetyLoc()
{
return yLocation;
}
char GetCh()
{
return chShow;
}
}

I look pretty pissed in this video – its probably due to inclement weather.

My implementation of Text Rain is fairly straightforward. The code is probably longer than needed for the simple effect, but I added some things, and wrote it because I thought this was something that could be made better in the future. One thing I really want to do is add awesome lightning effects that I can control with my hands.

Each droplet is an object with position variables and it’s derivatives. Every run through the draw loop, these are updated according to the preset numbers. Since I have acceleration in my droplets, it was important for me to limit the droplet speed using a terminal velocity. Finding these numbers was done by trial and error, and no doubt there is still room for improvement.

/* Droplets fall from the top of the screen and stop when they reach a certain threshold */
class Droplet{
//The character we are drawing
char c;
//The font size the droplet is being drawn
int size;
//The current position, velocity, and acceleration of the drop
float x, y, x_v, y_v, x_a, y_a;
//The starting position, velocity, and acceleration of the drop
int init_x, init_y, init_x_v, init_y_v, init_x_a, init_y_a;
//The width and height of the
int width, height;
//Terminal velocity for the droplet
//Change this value if you want it to snow instead
int terminal_v = 12;
//Whether this droplet is now snow
boolean isSnow;


By default droplets have an alpha value, and so, drops get darker when they are stacked. This made sense to me, since I think of the droplets as combining like they would in real life.

Aside from what Camille Utterback and Romy Achituv made, my implementation allows you to control the direction of the wind (using left and right keyboard) so that droplets move horizontally as well. Since it is winter, I also added a snow toggle (press shift), which can slow down the droplets. All of this is possible by simply setting the properties of all Droplet objects.

GitHub: https://github.com/dgurjar/Text-Rain

Originally by Camille Utterback and Romy Achituv, I was able to run “Text Rain” in Processing. It was my first time back to Processing in a while, but I was able to figure it out after some help and practice. I simple indexed the camera’s pixels and went about it the brightness() way. Fairly reliable and quite a lot of fun getting it going. I think a performance of my lip-singing my favorite song is in order for a finished piece.

Text Rain – Re Do from Nathan Trevino on Vimeo.

My code is here.

//Nathan Trevino 2013
//Text Rain re-do. Original by Camille Utterback and Romy Achituv 1999
//Processing 2.0b7 by Nathan Trevino
//Special thanks to the processing example codes (website) as well as Golan Levin

//=============================================
import processing.video.*;
Capture camera;

float fallGravity = 1;
float fallStart = 0;

int threshold = 100;
Rain WordLetters[];
int myLetters;


//==============================================

void setup() {

  //going with a larger size but am giving up speed.
  size(640, 480);



  camera = new Capture(this, width, height);
  camera.start();     



  String wordString = "For all the things he could lose he lost them all";

  myLetters = wordString.length();
  WordLetters = new Rain[myLetters];
  for (int i = 0; i < myLetters; i++) {
    char a = wordString.charAt(i);
    float x = width * ((float)(i+1)/(myLetters+1));
    float y = fallStart;
    WordLetters[i] = new Rain(a, x, y);
  }

}

//==============================================


void draw() {
  if (camera.available() == true) {
    camera.read();
    camera.loadPixels();

    //Puts the video where it should be... top left corner beginning.
    image(camera, 0, 0);

    for (int i = 0; i < myLetters; i++) {
      WordLetters[i].update();
      WordLetters[i].draw();
    }
  }
}

//===================================
//simple key pressed fuction to start over the Rain

void keyPressed()
{
  if (key == CODED) {
    if (keyCode == ' ') {
      for (int i=0; i < myLetters; i++) {
        WordLetters[i].reset();
      }
    }
  }
}


//=============================================
class Rain {
  // This conains a single letter of the words of the entire string poem
  // They fall as "individuals" and have their own position (x,y) and character (char)

  char a;
  float x;
  float y;

  Rain (char aa, float xx, float yy)
  {
    a = aa;
    x = xx;
    y = yy;
  }

  //=============================================
  void update() {
    
    //IMPORTANT NOTE!
    // THE TEXT RAIN WORKS WITH A WHITE BACKGROUND AND THE DARK AREAS 
    // MOVE THE TEXT

    // Updates the paramaters of Rain
    // had some problems here for the pixel index, but a peek at Golan's code helped

    int index = width*(int)y;
    index = constrain (index, 0, width*height-1);

    // Grayscale starts here. Range is defined here.
    int thresholdGive = 4;
    int thresholdUpper = threshold + thresholdGive;
    int thresholdBottom = threshold - thresholdGive;


    //find pixel colors and make it into brighness (much like alpha channeling video
    // or images in Adobe photoshop or AE)

    float pBright = brightness(camera.pixels[index]);

    if (pBright > thresholdUpper) {
      y += fallGravity;
    } 
    else {
      while ( (y > fallStart) && (pBright < thresholdBottom)) {
        y -= fallGravity;
        index = width*(int)y + (int)x;
        index = constrain (index, 0, width*height-1);
        pBright = brightness(camera.pixels[index]);
      }
    }

    if ((y >= height) || (y < fallStart)) {
      y = fallStart;
    }
  }

  //============================
  void reset() {
    y = fallStart;
  }

  //=======================================

  void draw() {

    // Here I also couldn't really see my letters that well so I
    // used Golan's "drop shadow" idea and some crazy random colors for funzies

    fill (random(255), random(255), random(255));
    text (""+a, x+1, y+1);
    text (""+a, x-1, y+1); 
    text (""+a, x+1, y-1); 
    text (""+a, x-1, y-1); 
    fill(255, 255, 255);
    text (""+a, x, y);
  }
}

For this implementation of Textrain I made a Letter class and kept track of an array of Letter objects. Each character from the string is stored in this array and updated via for loop in the draw function. They detect the brightness of the pixel immediately beneath them and only drop if the pixel is bright. There is also a function that fades the letters from teal to navy blue over time. The string is the first line of the e.e. cummings poem ‘anyone lived in a pretty how town,’ which really doesn’t have any significance other than the fact that it’s great.

GitHub -> soon, having trouble with it currently…

Code!

import processing.video.*;
Capture cam;

int height = 480;
int width = 640;

int g = 0;
int b = 100;
boolean flipG = false;
boolean flipB = true;

class Letter {
  int y;
  int x;
  int speed;
  char c;

  Letter(char character, int xPos, int yPos) {
    c = character;
    x = xPos;
    y = yPos;
    speed = (int) random(1, 2);
  }
}

ArrayList currentLetters;
Letter testLetter = new Letter('a', 50, 50);

void setup() {
  size(width, height);
  background(255);
  frameRate(10);

  textSize(18);

  cam = new Capture(this, width, height);
  cam.start();

  smooth();

  currentLetters = new ArrayList();
  String poem = "anyone lived in a pretty how town (with up so floating many bells down)";

  int xInit = 0;
  int yInit = 0;

  for (int i=0; i < 71; i++) {
    if (i<33) yInit = (int)random(-10, 10);
    else yInit = (int)random(-15, -25);
    currentLetters.add(new Letter(poem.charAt(i), xInit, yInit));
    xInit += (9 + (int)random(-3, 3));
    if (xInit > width) xInit = 9;
  }
}

void draw() {
  cam.read();
  cam.loadPixels();

  pushMatrix();
  translate (width, 0); 
  scale(-1, 1);
  image (cam, 0, 0, width, height);
  popMatrix();

  fill(0, g, 100);

  for (int i=0; i < 71; i++) {
    Letter curr = currentLetters.get(i);

    int index = width*curr.y + (width-curr.x-1);
    index = constrain (index, 0, width*height-1);

    if ((brightness(cam.pixels[index])) > 105) curr.y += curr.speed;
    else {
      while ((curr.y > 0.0) && ((brightness(cam.pixels[index])) < 95)) {
        curr.y -= .025;
        index = width*curr.y + (width-curr.x-1);
        index = constrain (index, 0, width*height-1);
      }
    }
    text(curr.c, curr.x, curr.y);

    if (curr.y >= height) {
      curr.y = (int)random(-15, 15);
      curr.speed = (int)random(1, 2);
    }
  }

  if(!flipG) {
    if (g<100) g++;
    else flipG=true;
  }
  else if(flipG) {
    if (g>0) g--;
    else flipG=false;
  }

  if(!flipB) {   
    if (b<150) b++;
    else flipB=true;
  }
  else if(flipB) {
    if (b>0) b--;
    else flipB=false;
  }
}

Punctuation Rain from Mike Taylor on Vimeo.

This is a basic reimplementation of Text Rain by Camille Utterback and Romy Achituv.  The Processing code is relatively efficient as it extracts brightness values directly from the camera pixels that have already been written to the screen.  It also only examines pixels at the current location of the falling letters, rather than performing operations over the entire image.  As with most simple implementations, the app relies on a light-colored background to work properly.  The letters fall at a speed proportional to the brightness difference between the pixel and the light/dark threshold.  Below that threshold, the letters rise until finding a light region again, resulting in the apparent tendency of the letters to “ride” arms and other moving objects.  Additional techniques like background subtraction or true boundary detection could improve performance.

The Processing code can be found here.

And here:

import processing.video.*;

Capture cam;

int xp = 1;
char ch;
int spacing = 30;

//String str = "***************************************";
String str = "!@#$%^&*(^%$##$(#@@#$%&!@#$%^&^%$)%#^&*";
int[] chary = new int[str.length()];

void setup() {
size(1280,720);

textSize(32);

for(int i=0; i<chary.length; i++){
chary[i]=0;
}

String[] cameras = Capture.list();

if (cameras.length == 0) {
println("There are no cameras available for capture.");
exit();
} else {
println("Available cameras:");
for (int i = 0; i < cameras.length; i++) {
println(cameras[i]);
}

// The camera can be initialized directly using an 
// element from the array returned by list():
cam = new Capture(this, cameras[0]);
cam.start(); 
} 
}

void draw() {
if (cam.available() == true) {
cam.read();
}
image(cam, 0, 0);
// The following does the same, and is faster when just drawing the image
// without any additional resizing, transformations, or tint.
//set(0, 0, cam);

for (int i = 0; i < str.length(); i=i+1) {
ch = str.charAt(i);
chary[i]=chary[i]+int((brightness(get((i+1)*spacing,chary[i]))-80)/20);
if((chary[i]>720)||(chary[i]<0)) {
chary[i]=0;
}
fill(0,102,153);
text(ch,(i+1)*spacing,chary[i]);

}

//println((brightness(get(mouseX,mouseY)))-100);
}