as an example, here's how i do it with flash. listen to incoming audio in pd and create forms, colors and mouvements in flash. no interaction is required from user : http://www.pure-data.org/community/patches it's called pure flash
you will need flashserver from olaf : http://www.akustische-kunst.org/puredata/flash/flash.html
work best with electro funk music or voice ;) patrick