Catching Flies

Using a Trampoline and Microsoft Kinect

Trampolines offer a new degree of freedom while interacting with the Kinect, and I wanted to create a game that demonstrated how one could be used to enhance the Kinect experience. Unlike regular jumping, a trampoline creates a regular and more predictable rhythm of vertical motion and can be sustained for a longer period of time.

Starting with that concept I originally began to create a side-scrolling game similar to Super Mario Brothers that would have the user jump to hit boxes and jump over pipes and enemies. I abandoned this attempt when it was clear that
it was uncomfortable for the player to adjust their actions to one that affected an orthogonal vertex. Additionally, complex object intersection became arbitrary complexity for a project attempting to achieve a proof of concept of rhythm
detection, skeleton tracking while bouncing, and hop detection.

Frog Arms

In the next iteration, the player was a frog whose arms would reflect their relative vertical position. A spring force at the end of each hands directs upward motion slightly outward and creates the illusion that the player is a frog jumping forward at each hop.

Hop Detection

The most interesting phase of the project involved player hop detection. One full cycle of ascending and descending could be broken down into logical phases of a bounce. Using the middle of the shoulders as the most stable part of the body from which to determine the acts of ascending and descending, every update cycle would pull that vertical position and place it into a height cache. The height cache was an array of height values treated as a sliding window: Instead of comparing the current y position to the most recent y position it was compared to some defined number of former y positions. As the current y position would have to be larger than all the former readings within the window, the sliding window reduced the likelihood of a bad reading or jitter causing a false determination that the player has started a new phase of ascending or descending. After a period of ascension, the player is considered in stasis when a future y position fails to be larger than all members of the height cache.

This is reflected in the following logic:

bool asc = true;
bool desc = true;
for( int i=0; i < HEIGHT_CACHE_SIZE; i++) {
    float old = properties.heightCache[i];
    //height pulled from tracked user is relatively inversed
    if(old > y) {
        desc = false;
    }
    else if(old < y) {
        asc = false;
    }
}
 
// if we are either ascending or descending (not in stasis)
// update the previous state
if( asc || desc) {
    // on first time leaving stasis; preserve former asc/desc states
    if(!properties.inStasis) {
        properties.wasAscending = properties.isAscending;
        properties.wasDescending = properties.isDescending;
    }
    properties.isAscending = asc;
    properties.isDescending = desc;
    properties.inStasis = false;
}
else {
    if (!properties.inStasis){
        properties.wasAscending = properties.isAscending;
        properties.wasDescending = properties.isDescending;
        properties.isAscending = false;
        properties.isDescending = false;
        properties.inStasis = true;
    }
}

Finally, with this logic in place, a hop is accounted for by the following:

if( wasAscending() and isDescending() and 
        (abs(y - properties.average_weighted_y) > REST_Y_DELTA)) {
    properties.num_hops++;
}

NB: The REST_Y_DELTA is used to account for the trampoline’s affect on a player’s rest state.

Gameplay – Flies

With the foundation in place, gameplay became the next phase of development. Continuing with the frog theme, I introduced flies as a desired token within the game. As the arms were presently unused, they became the tool used for fly interaction. Once the fly came close enough (they would gradually become larger as they approached the player), the player could wave their hand above shoulder level to swipe the frog’s tongue across the screen and gather any flies that hadn’t escaped off screen.

Gameplay – Stats

In order to show a player’s progress and bring encouragement, their number of hops and the number of flies their frog had collected were displayed on clouds at the top of the screen.

Gameplay – Endgame

Finally, the end game became the grass descending into the sea. Each hop would keep the grass from descending some marginal amount increasing the length of the game and thus the time available to catch flies.

Conclusion

While gameplay left a lot to be desired, I believe playing to my strengths and focusing on logical determinations of the jumper’s state and the resulting generic ‘jumper’ class and header files, which can be dropped in to future games involving rhythmic vertical motion, will ultimately prove worthwhile. After the public exhibition, it became clear that smaller children were having trouble getting proper hop detection and corresponding calculations. Once I fix some of the code that relied on incorrect constant values, it should be ready for general release.

Icon & One Sentence Description

Fly Catching is a game using a Kinect and trampoline where the player controls a frog that can hop and catch flies.