Oh man… here wei go. I like the graphic on your napkin sketch of the ball rolling over the hills-seems like an interesting approach and I haven’t seen it before. It’d be fun to set up jumps and ramps along the hills that added complexity to the music you were outputting. I’d agree with the other comments on here that actually generating the sound could be pretty challenging, but it sounds like you have friends in high places that might be able to help out…
Also I would vote.. OMG NOT MOBILE. That would be an utter disaster. The real-time audio generation APIs on the iPhone are a hairy mess of C. You don’t get to use the high level Objective-C APIs unless you’re doing playback of pre-recorded audio. Or you can use libpd on iOS … :D
There are plenty of precendents out there to check out, including in the iOS app store. Creating a realtime visual score/instrument is not an easy task actually but there are natural mappings we are already used to you can use. The trick is how to create something that works for you out of it.
Coincidentally, a group I usually work with is making something similar. We have a game going with the hills generated from music data. Let me know if you want to see some of their code / use it as a starting point. I think they’re writing it in C++ -Susan
Sound in Processing: Minim should already be installed, that does some fun things. Take a look at FFT and spectrum analysis for examples of common visualizations.
Look at everything by Toshio Iwai (Electroplankton), Zach Gage,
Balldroppings by Josh Nimoy
Golan says: please remind me to show you Electroplankton
These are big questions you’re asking the group; you’ll need to focus by doing some more research and then.. just picking something
I think that’s not going to be easy. You’re going to meet with a lot of ambiguity and make decisions which may or may not be aurally pleasing. Or simply just getting too much data.
Could you have music representing some sort of gameplay? Perhaps a simple game like Tetris, or even something like Robot Unicorn Attack
Is the goal more to create good music or fun interactions?
Careful though, in my personal experience unless you have some pretty strict rules on the octaves/ranges, generative sound is difficult to make sound…pleasing. Its much easier to go the other way, but that’s been done time and again.
Minim for processing is easy to use out of the box.
I’m no expert in music representation, but I feel like synchronization is very very important for a coherent interface.
You may want to start by coming up with a infoviz or something that you can then turn into music… It may be a way to approach the problem without getting into strict music visualization. You might find the result a little more interesting.
Maybe it could be like one of the examples you have: create something random or interesting graphically, then let the program create music from that once you hit play.
Check out Singing Fingers for iPhone
ambitious & hard!
Is there a particular way you want to enable users to create music?
Here are today’s comments:
Oh man… here wei go. I like the graphic on your napkin sketch of the ball rolling over the hills-seems like an interesting approach and I haven’t seen it before. It’d be fun to set up jumps and ramps along the hills that added complexity to the music you were outputting. I’d agree with the other comments on here that actually generating the sound could be pretty challenging, but it sounds like you have friends in high places that might be able to help out…
Also I would vote.. OMG NOT MOBILE. That would be an utter disaster. The real-time audio generation APIs on the iPhone are a hairy mess of C. You don’t get to use the high level Objective-C APIs unless you’re doing playback of pre-recorded audio. Or you can use libpd on iOS … :D
There are plenty of precendents out there to check out, including in the iOS app store. Creating a realtime visual score/instrument is not an easy task actually but there are natural mappings we are already used to you can use. The trick is how to create something that works for you out of it.
See:
– The Visual Context of Music: http://bibliodyssey.blogspot.com/2006/07/visual-context-of-music.html
– The Tenori On: https://www.youtube.com/watch?v=_SGwDhKTrwU
– The Otomatone: https://www.youtube.com/watch?v=qecPQ0FB-64&feature=related
– The Monome: https://www.youtube.com/watch?v=cJwxbTKwONc&feature=related
– Stockhausen on Sound: https://www.youtube.com/watch?v=pIPVc2Jvd0w
– Iannis Xenakis Metastasis: https://www.youtube.com/watch?v=SZazYFchLRI&feature=related
There is also a computer music/music tech group at CMU headed by Roger Dannenberg in the CS department.
Here’s my thesis (focused on digital instrumentation): http://danomatika.com/media/projects/s2007/thesis/dwilcox_thesis_arttech_07.pdf
Maybe Brian Eno’s generative iphone stuff could be an inspiration: http://www.generativemusic.com/
Check out bloopsaphone: https://github.com/atduskgreg/bloopsaphone
Coincidentally, a group I usually work with is making something similar. We have a game going with the hills generated from music data. Let me know if you want to see some of their code / use it as a starting point. I think they’re writing it in C++ -Susan
Sound in Processing: Minim should already be installed, that does some fun things. Take a look at FFT and spectrum analysis for examples of common visualizations.
Look at everything by Toshio Iwai (Electroplankton), Zach Gage,
Balldroppings by Josh Nimoy
Golan says: please remind me to show you Electroplankton
These are big questions you’re asking the group; you’ll need to focus by doing some more research and then.. just picking something
Read master’s theses by Golan Levin and Dan Wilcox
http://acg.media.mit.edu/people/golan/thesis/
I think that’s not going to be easy. You’re going to meet with a lot of ambiguity and make decisions which may or may not be aurally pleasing. Or simply just getting too much data.
Could you have music representing some sort of gameplay? Perhaps a simple game like Tetris, or even something like Robot Unicorn Attack
Is the goal more to create good music or fun interactions?
Careful though, in my personal experience unless you have some pretty strict rules on the octaves/ranges, generative sound is difficult to make sound…pleasing. Its much easier to go the other way, but that’s been done time and again.
Minim for processing is easy to use out of the box.
I’m no expert in music representation, but I feel like synchronization is very very important for a coherent interface.
You may want to start by coming up with a infoviz or something that you can then turn into music… It may be a way to approach the problem without getting into strict music visualization. You might find the result a little more interesting.
Maybe it could be like one of the examples you have: create something random or interesting graphically, then let the program create music from that once you hit play.
Check out Singing Fingers for iPhone
ambitious & hard!
Is there a particular way you want to enable users to create music?