süss

süss is a sound generator where the user creates lines that play sounds when strummed. The user controls the length and color of each line, which in turn affects the pitch and voice of the sound it makes. By putting lines near each other, chords and simple melodies can be created. Currently, the project is controlled using a trackpad and supports a few different multi-touch interactions, so it can easily be adapted to touchscreen devices such as the iPad or iPhone in the future.

INSPIRATION
Since I have never worked with audio in any projects before, I decided this would be a good time to learn. I was inspired by a few projects which incorporated music and sound with a visual representation of what was being heard. In some projects, the sound was support for the visuals (or vice versa), but in other projects the two aspects were combined quite seamlessly. I was especially interested in applications that allowed users to create their own music by making something graphical that then gets translated into audio in a way that looks good, sounds good, and makes sense.

mta.me is actually very similar to what I ended up doing for this project, although I only realized this just now when looking back on my research. This is a visualization of the NYC subway schedule, and every time a line is crossed by another it gets plucked and makes a sound.

A couple of other, more painting based projects that I looked at:

PROCESS

I knew from the beginning that I wanted to make a visual tool for creating music. However, I didn’t really know what a good way to represent the sounds would be, and I didn’t know how to handle playback of the audio. So I did some sketching to get some initial ideas flowing.

I also set up some code using TrackPadTUIO [http://www.010175.net/?browse=Monolithe], which gave me a way to access what the trackpad was recording. This got me started on thinking about ways to finger paint with sound.

I also chose to use maximilian [http://www.maximilian.strangeloop.co.uk/] to synthesize sounds. I had a lot to learn in this area, so in the beginning stages I only had really simple sine waves or really ugly computery sounds. Nevertheless, I gave the user the option of four sounds represented by paint buckets, that they could dip their fingers into and paint with around the trackpad while the sounds played. I decided to have the y-position of the fingers control the pitch of the sounds, but I wasn’t really sure what to do with the x-position, so I tried out tying it to some filter variables and some other random stuff. Eventually, I got tired of playing around with such ugly noises and decided to see if I could make real notes, so I implemented a little touchpad keyboard. It sounded okay, but it really wasn’t what I was going for in my project, so I eventually got rid of it. However, the keyboard was a good exercise in working with real notes, which made its way into the final iteration of my project. Even so, I was pretty unhappy with my progress up until a couple of days before the final show.

Two nights before the final show, I decided to scrap most of what I had and switch to a new concept, which ended up being süss.

My main motivation for switching concepts was to simplify the project. Before this, I had too many sounds and visuals going on with all the paint trails and long, sustained notes. By having the user create areas that get activated through their own actions, sounds are only played every once in a while, and we get closer to features such as rhythms and melodies. Another success was when I figured out how to use envelopes with the maximilian library, which led to slightly more pleasant sounds, although they are still not great. With help from my peers, I was also able to give the project a facelift so that it looks much better than what I had in my intermediate stages.

REFLECTION
From the beginning of my project up until the very last days, I only had a general idea of what I wanted to do. A big problem throughout the process was my indecision and lack of inspiration in figuring out exactly what would be a good visualization and interaction for my project. At my in class presentation, I was still not happy with what I had, and I had to do a lot of work to turn it around in time for the final.

I’m pretty satisfied with what I have now, but there are definitely areas that could be refined. I think the project could be really helped by adding animation to the strings when they get plucked. As always, the different voices could be improved to sound more beautiful, which would contribute to making nicer sounding compositions. And the way I implemented it, it would really be more appropriate for something with a touchscreen, so that your fingers are actually on top of the lines you are creating.

I learned a lot from this experience, especially in the realm of audio synthesis. Before this project, I had no idea what waveforms, envelopes, filters, and oscillators were, and now I at least vaguely understand the terms (it’s a start!). I also know how hard it is to make sounds that are decent, and how easy it is to write a bug that almost deafens me when I try to run my code.

100×100 and quick summary

süss is an interactive sound generator using touch interactions to create strings that can be “strummed”.