Direct selection: it’s not clear which brush you’re actually editing. “Brush #5” is not very helpful. It would also be helpful if they were actually animating (perhaps in a scratch pad) while you were editing their properties.

Not offense, but the colors are a little crude. One common idea used by a lot of designers (Casey Reas, Erik Natzke — see his Flickr site) is to sample colors from a pre-loaded natural image.

Is there a way to smooth the lines? Right now it just doesn’t look particularly brush-like.

It should show the resulting color more clearly. It’s sometimes difficult to remember how colors mix when selecting your colors. Maybe do the hue circle thing.

Selecting visually pleasing color combinations would help you a lot here too. There are some very simple calculations you can do with a color wheel to make nice combinations. Just convert into HSV color space and then shift 120º each time for triad colors. See: http://en.wikipedia.org/wiki/Color_theory

I think you need maybe an RGB map that you can select from, then a box that shows what the current color is. Also, being able to set background color might be nice so that things can start to pop.

Undo is fun to implement, though at this point, it will likely involve gutting a lot of your current code. It involves making “actions” out of everything you do. So that every time you draw, you create a new ‘drawAction’ that keeps track of where and what you drew. Undo and Redo have been highly architected, you should be able to find some examples.
Undo is a pretty important feature to have, because it empowers users to play around with the interface without having to worry that they will break something.

Can you erase a line?

Can you add the new point to a curve that you store as an object, so that when you redraw them, each line is made in its entirety before the next is created. I think this would get rid of your layeing issue.

Instead of a ‘testing area’ you could show a simple representation stroke like this:
http://www.brushthis.com/wp-content/uploads/2008/07/dynamic-brushes.jpg