Hi, I've been thinking about using PD and GEM in CAVE-like immersive virtual environments like the PORTAL at TU Berlin and the CAVE/CUBE at UIUC (www.math.tu-berlin.de/geometrie/f5/portal.shtml, www.isl.uiuc.edu). Has this been done before? I did find a few passing references to PD and sound in CAVEs on the web, but nothing about PD and GEM.
The basic setup of a CAVE consists of a number of projection screens, aka walls (three for the PORTAL, six for the CUBE), controlled by one computer each, plus one master computer. Here's what I have in mind: - The master does audio with PD and handles user input via mouse, keyboard, MIDI controller, etc. - The wall computers run visualizations with PD and GEM. Those visualizations are identical except they use different viewports for different walls. - The master controls the walls via netsend/netreceive. - A python script on the master handles CAVE-specific input (such as head tracking data) and updates the master via pdsend/pdreceive. It may be a good idea to build a little python extension that encapsulates the functionality of pdsend/pdreceive.
All this seems straightforward enough, but there are two issues I'm worried about: - Can I change the viewport in GEM? The FAQ has some comment regarding the default viewport, but little information on how to change it. - How well does GEM support stereo viewing?
Any thoughts would be appreciated! Best, Peter
Hi, I've been thinking about using PD and GEM in CAVE-like immersive virtual environments Any thoughts would be appreciated!
-I seem to remember that Søren Bovbjerg at CVMT is involved with both matters, To what extent they are combined or combinable, I don't know, but you might go to: http://www.cvmt.dk/~sb/ and see what the state of affair is....
AvS
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .....................................................................
` |Schreck Ensemble . . . . . . . . . . . . . . . . . . . . +
` |# -laboratory for live electro-acoustic music- # |
| http://www.schreck.nl/ |
| http://www.xs4all.nl/~schreck/ |
` *===========================================================++
` |Compositions http://www.xs4all.nl/~schreck/html/compo.html |
` |Samples http://www.xs4all.nl/~schreck/html/samp.html |
` |Patches http://www.xs4all.nl/~schreck/html/pat.html |
` |Videos http://www.xs4all.nl/~schreck/html/video.html |
` |Scores http://www.xs4all.nl/~schreck/html/scores.html |
*===========================================================++
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .....................................................................
Arie, Thanks for your reply! I checked out Soren's homepage, but it appears that his VR projects and PD projects are disjoint. I guess I'll get in touch with him directly and find out what's up. Best, Peter
On Sat, Jan 22, 2005 at 10:04:41PM +0100, Arie van Schutterhoef wrote:
Hi, I've been thinking about using PD and GEM in CAVE-like immersive virtual environments Any thoughts would be appreciated!
-I seem to remember that Søren Bovbjerg at CVMT is involved with both matters, To what extent they are combined or combinable, I don't know, but you might go to: http://www.cvmt.dk/~sb/ and see what the state of affair is....
AvS
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .....................................................................
` |Schreck Ensemble . . . . . . . . . . . . . . . . . . . . +
` |# -laboratory for live electro-acoustic music- # | | http://www.schreck.nl/ | | http://www.xs4all.nl/~schreck/ | ` *===========================================================++ ` |Compositions http://www.xs4all.nl/~schreck/html/compo.html | ` |Samples http://www.xs4all.nl/~schreck/html/samp.html | ` |Patches http://www.xs4all.nl/~schreck/html/pat.html | ` |Videos http://www.xs4all.nl/~schreck/html/video.html | ` |Scores http://www.xs4all.nl/~schreck/html/scores.html | *===========================================================++. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .....................................................................
Hi,
i've been thinking about this too, but had no time to start it.
- Can I change the viewport in GEM? The FAQ has some comment regarding
the default viewport, but little information on how to change it.
the helpfile Gem/gemwin.pd says some details in the viewing section:
messages to [gemwin] regarding the view-point ("camera"): perspec <left> <right> <bottom> <top> <front> <back>
set the clipping planes of the view-point. this might be what you need , if all the objects that re further away than 20 units suddenly disappear; default: -1 1 -1 1 1 20 view <x> <y> <z>
translate the camera / set the viewpoint:: the viewing-direction will not be changed.; default: 0 0 4 the viewing direction defaults to "0 0 1" , with y-axis as "up" view <x> <y> <z> <azimuth> view <x> <y> <z> <azimuth> <elevation> translate/rotate camera/viewpoint ; the "up" direction will still be the y-axis view <view_x> <view_y> <view_z> <target_x> <target_y> <target_z> <up_x> <up_y> <up_z>
set viewpoint (view_X view_Y view_Z). the camera will look at the target-point (target_X , target_Y , target_Z). "up" is defined via the vector (up_X , up_Y , up_Z) default: 0 0 4 0 default: 0 0 4 0 0 0 0 1 0
if you want to project on uneven surfaces (like a dome, a planetary, an edge or a cave) you might be interested in http://tot.sat.qc.ca/eng/lighttwist.html http://vision3d.iro.umontreal.ca/
http://astronomy.swin.edu.au/~pbourke/projection/ http://astronomy.swin.edu.au/~pbourke/projection/3walls/ http://astronomy.swin.edu.au/~pbourke/projection/cylinder/
On Tue, Jan 25, 2005 at 05:53:51PM +0100, Max Neupert wrote:
i've been thinking about this too, but had no time to start it.
Hmm, maybe we should talk some time... My main motivation for contemplating GEM in virtual environments is to open our existing installation at TU Berlin (the PORTAL - a CAVE-like environment with three walls) for collaboration with artists. What sort of installation are you going to target?
- Can I change the viewport in GEM? The FAQ has some comment regarding
the default viewport, but little information on how to change it.
the helpfile Gem/gemwin.pd says some details in the viewing section: [...]
Excellent, that's precisely what I was looking for!
if you want to project on uneven surfaces (like a dome, a planetary, an edge or a cave) you might be interested in http://tot.sat.qc.ca/eng/lighttwist.html http://vision3d.iro.umontreal.ca/
Interesting stuff! For the time being, all the screens within my reach are quite flat, but I'll keep this in mind for future reference. Cheers, Peter
I gave a talk on GEM at the EVL at UIC here in Chicago and there was discussion about using it for such purposes. The large CAVE system still runs on a four pipe SGI Onyx system and that is primarily due to sync for the four displays rather than raw horsepower. The main obstacle to using multiple machines in the CAVE is the precise display sync for the four walls required for the full 3D illusion. The EVL has a new single wall dual projector Linux system that would be better suited to using Pd/GEM even if only as a bridge to the VRML CAVE environment. I'm not sure if anything has been done on the implementation there or not. I do know of projects outside the CAVE that mixed the VRML engine with Pd/GEM with great success.
I would be interested in looking into this again, and I believe some of the faculty and students at UIC read this list from time to time (Drew, Lief are you out there?). There's definitely some opportunities to explore.
cgc
On Jan 22, 2005, at 2:35 PM, Peter Brinkmann wrote:
Hi, I've been thinking about using PD and GEM in CAVE-like immersive virtual environments like the PORTAL at TU Berlin and the CAVE/CUBE at UIUC (www.math.tu-berlin.de/geometrie/f5/portal.shtml, www.isl.uiuc.edu). Has this been done before? I did find a few passing references to PD and sound in CAVEs on the web, but nothing about PD and GEM.
The basic setup of a CAVE consists of a number of projection screens, aka walls (three for the PORTAL, six for the CUBE), controlled by one computer each, plus one master computer. Here's what I have in mind: - The master does audio with PD and handles user input via mouse, keyboard, MIDI controller, etc. - The wall computers run visualizations with PD and GEM. Those visualizations are identical except they use different viewports for different walls. - The master controls the walls via netsend/netreceive. - A python script on the master handles CAVE-specific input (such as head tracking data) and updates the master via pdsend/pdreceive. It may be a good idea to build a little python extension that encapsulates the functionality of pdsend/pdreceive.
All this seems straightforward enough, but there are two issues I'm worried about: - Can I change the viewport in GEM? The FAQ has some comment regarding the default viewport, but little information on how to change it. - How well does GEM support stereo viewing?
Any thoughts would be appreciated! Best, Peter
PD-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://iem.at/cgi-bin/mailman/listinfo/pd-list
-- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. MailScanner thanks transtec Computers for their support.
The large CAVE system still runs on a four pipe SGI Onyx system and that is >primarily due to sync for the four displays rather than raw horsepower. The main obstacle to using multiple machines in the CAVE is the precise display sync for the four walls required for the full 3D illusion.
-Irix is still a nice OS...
AvS
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .....................................................................
` |Schreck Ensemble . . . . . . . . . . . . . . . . . . . . +
` |# -laboratory for live electro-acoustic music- # |
| http://www.schreck.nl/ |
| http://www.xs4all.nl/~schreck/ |
` *===========================================================++
` |Compositions http://www.xs4all.nl/~schreck/html/compo.html |
` |Samples http://www.xs4all.nl/~schreck/html/samp.html |
` |Patches http://www.xs4all.nl/~schreck/html/pat.html |
` |Videos http://www.xs4all.nl/~schreck/html/video.html |
` |Scores http://www.xs4all.nl/~schreck/html/scores.html |
*===========================================================++
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .....................................................................
Chris, Thanks for your reply!
I gave a talk on GEM at the EVL at UIC here in Chicago and there was discussion about using it for such purposes. The large CAVE system still runs on a four pipe SGI Onyx system and that is primarily due to sync for the four displays rather than raw horsepower.
Hmm, I thought that EVL had dismantled its CAVE. At least, I've been told that CAVE components from UIC are currently sitting in some basement at UIUC, awaiting reassembly. When did you give your talk?
The main obstacle to using multiple machines in the CAVE is the precise display sync for the four walls required for the full 3D illusion.
Sure. However, the installations I'm involved with (PORTAL and CUBE) have fast machines and fast networks, and the sort of applications I have in mind wouldn't have to move an awful lot of data from the master to the walls. That, combined with the low temporal resolution of the human eye, gives me the impression that my idea may be viable. Alternatively, the master copy of PD could send out clock messages to synchronize rendering.
For a single wall, it should certainly work, and even that would be interesting. Plus, the setup I described shouldn't take more than an afternoon or so to build. After all, it would hardly involve any programming; just take some existing components and put them together. If synchronization turns out to be problematic, fine, I'll scale it down to a single wall.
I do know of projects outside the CAVE that mixed the VRML engine with Pd/GEM with great success.
Any pointers?
I would be interested in looking into this again, and I believe some of the faculty and students at UIC read this list from time to time (Drew, Lief are you out there?). There's definitely some opportunities to explore.
Excellent! This might be a very worthwhile project. While there's no shortage of packages that drive CAVEs (CAVE libs, Syzygy, VR++, jReality, etc.), there's relatively little software that's built upon them. Putting PD and GEM in virtual environments should open up a plethora of potential CAVE applications, and it might enable collaboration with a whole new group of people. Peter
Hello
I showed on 22. in Berlin a VR installation (but not Cave, just one screen and footcontrolled input) done with another open source free software called Blender
Its a 3D modelling and animation package which includes a Realtime Engine which also can save standalone applications, is scriptable with Python and can do Stereorendering:
http://www.blender3d.org/cms/Game_engine.487.0.html
Cheers,
Malte
Malte Steiner media art + development -www.block4.com-
Peter Brinkmann wrote:
- How well does GEM support stereo viewing?
right now, there are 2 stereo-modes in Gem, anaglyph (e.g. red/green) and one with 2 separate sub-windows (both in one "real" window).
anaglyph renderign is probably not what you want.
Any thoughts would be appreciated!
btw, the multiple_window feature could likely be what you need for a cave-like situation. in theory you could have one instance of Gem on one master-computer, and several slave-renderers (via remote windows) this would (right now) only work under linux and i am pretty sure that you could not deal with large textures. (i once did it with live-video forwarded to 1 single slave, which worked, but never tried several slave-machines (multiple_window did not exist back then)
mfg.a.dsr IOhannes
On Mon, Jan 31, 2005 at 08:56:35AM +0100, IOhannes m zmoelnig wrote:
Peter Brinkmann wrote:
- How well does GEM support stereo viewing?
right now, there are 2 stereo-modes in Gem, anaglyph (e.g. red/green) and one with 2 separate sub-windows (both in one "real" window).
Having two subwindows would work; the graphics hardware in the PORTAL supports both twin mode and quad mode. The subwindow option would work in twin mode. A quad option would be great to have, but twin will do.
btw, the multiple_window feature could likely be what you need for a cave-like situation. in theory you could have one instance of Gem on one master-computer, and several slave-renderers (via remote windows) this would (right now) only work under linux and i am pretty sure that you could not deal with large textures.
This sounds interesting, but I'm a bit worried here. When you say 'remote windows', do you mean X forwarding? My experience is that X forwarding in the PORTAL slows things down quite a bit. Best, Peter
Peter Brinkmann wrote:
btw, the multiple_window feature could likely be what you need for a cave-like situation. in theory you could have one instance of Gem on one master-computer, and several slave-renderers (via remote windows) this would (right now) only work under linux and i am pretty sure that you could not deal with large textures.
This sounds interesting, but I'm a bit worried here. When you say 'remote windows', do you mean X forwarding? My experience is that X forwarding in the PORTAL slows things down quite a bit.
yes, it is kind of X-forwarding; my guess was, that since X is forwarding only the openGL-instructions (instead of the window "pixel" contents) this should be fast enough (except when big textures are involved, where we are at the stage of normal pixel-forwarding)
mfg.asd.r IOhannes
Out of curiosity, how does one enable this second option (2 separate sub windows?)
A good friend of mine and I are working on a project (well, im helping him with some basic stuff..)... for doing 3D video playback from stereoscopic footage shot while skydiving. He has mounted 2 cameras to his helmet and has recorded some footage, and we are looking for easy ways to send frame accurate (read synched) video to two projectors, and will probably end up using polarized filters and glasses to view. We last tested max/jitter using a very large 1600x600 (2 800x600 quicktimes 'glued' together) and using jit.scissors to separate them to two instances of jit.window. Even on a dual G5 2.5 Ghz with 4 Gb ram, we dropped frames using mjpeg codec, but uncompressed video seemed to be ok. I would like very much to use Gem, but since one cant instantiate two windows (on OS X at least..), I thought it might not work... I dont think that G5 ever worked that hard, which to me was very very surprising.
Does anyone have any example patches doing something similar, and keeping proper frame synch from either one, or two pix_movies? I have great faith that Gem and PD will be much smoother.
Thanks much. You guys are all too cool!
These projects sound very interesting..
doktorp
http://homepage.mac.com/doktorp/ On Jan 31, 2005, at 2:56 AM, IOhannes m zmoelnig wrote:
Peter Brinkmann wrote:
- How well does GEM support stereo viewing?right now, there are 2 stereo-modes in Gem, anaglyph (e.g. red/green) and one with 2 separate sub-windows (both in one "real" window).
anaglyph renderign is probably not what you want.
Any thoughts would be appreciated!
btw, the multiple_window feature could likely be what you need for a cave-like situation. in theory you could have one instance of Gem on one master-computer, and several slave-renderers (via remote windows) this would (right now) only work under linux and i am pretty sure that you could not deal with large textures. (i once did it with live-video forwarded to 1 single slave, which worked, but never tried several slave-machines (multiple_window did not exist back then)
mfg.a.dsr IOhannes
PD-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://iem.at/cgi-bin/mailman/listinfo/pd-list
On Jan 31, 2005, at 6:21 PM, doktorp wrote:
We last tested max/jitter using a very large 1600x600 (2 800x600 quicktimes 'glued' together) and using jit.scissors to separate them to two instances of jit.window. Even on a dual G5 2.5 Ghz with 4 Gb ram, we dropped frames using mjpeg codec, but uncompressed video seemed to be ok. I would like very much to use Gem, but since one cant instantiate two windows (on OS X at least..), I thought it might not work... I dont think that G5 ever worked that hard, which to me was very very surprising.
The test build of GEM just posted allows for two windows with the same rendering context. I would think you could use this and either adjust the viewport or texture coordinates to split the frame in half. A single stereo view could work as well, but I don't have any experience with that.
The best way to keep sync would be to stitch the two frames into one 1600x600 in another application (AfterEffects?). I have used pix_movie on a dual 2 Ghz G5 to run 1920x1080 photo-jpeg without any problems, so I don't think you will have any trouble with your frame size. You could try two pix_movies and incrementing the frames using a counter, which would work in theory, but we all know how well theory works in practice.
cgc
On Feb 1, 2005, at 3:41 PM, chris clepper wrote:
The best way to keep sync would be to stitch the two frames into one 1600x600 in another application (AfterEffects?). I have used pix_movie on a dual 2 Ghz G5 to run 1920x1080 photo-jpeg without any problems, so I don't think you will have any trouble with your frame size. You could try two pix_movies and incrementing the frames using a counter, which would work in theory, but we all know how well theory works in practice.
...also need to make sure you have enough VRAM to do such large textures: use the function glGet* with GL_MAX_TEXTURE_SIZE as the parameter name to see what your hardware limit is...
jamie
-
Arie van Schutterhoef -
chris clepper -
doktorp -
IOhannes m zmoelnig -
james tittle -
Malte Steiner -
Max Neupert -
Peter Brinkmann