Hey all,
I've moved the discussion onto the gem-dev list. I think the pders have had enough of this discussion!
I did not see a link to any in depth documentation from chromium. It does sound very promising... I'm not a c++ programmer so I'm not sure what it would involve to build these functions into GL wappers for Gem. I'm not sure how the functions latch onto an existing context, and how the whole thing works architecturally (what parts run on what machines, master slave connections etc..
Since Gem is a library you can use gem-dependant objects anyware but inside Gem. Due to this I think all "externals" for Gem have been included in Gem proper. (why not?) The tiled display and multiple view stuff looks quite useful. Sebastien, do you think lighTWIST could end up being a set of objects for pd/Gem? Perhaps based on something like chromium (is it GPL?) plus your distortion stuff?
I just wonder about performance, considering the speed of the AGP bus for texture transfers vs ethernet transport between the source and destination for the texture! especially if your talking about moving video... (are you guys even using video in your cave application?)
Anyhow gem-devers this discussion does sound interesting, I'm sure some of you have ideas about Gem clustering for N projectors/machines.
B.
Mike Wozniewski wrote:
B. Bogart wrote:
The IDEAL solution would be a system that could take the RENDER data from the pixelTANGO gem-chains (well the whole context) pass the entire thing onto the multicast network and somehow dynamically build a copy of the pixelTANGO patch in a non-Gem host. That is you have a magic little application that just dynamically clones an exitsing GL context over network. Once we have the copy in the projector machine we can crop it for that projector, and apply the deformation grid. Of course references to pixel textures in memory, framegrabbers etc.. would not be interpreted propely. I don't see how chromium deals with this part of transporting the texture/video data...
Although I've just learned of Chromium, it sounds like it may be the ideal solution we're looking. From what I can tell, all that Chromium does is intercept openGL commands from arbitrary OpenGL applications, bundles them in a "stream", and passes them thru a 'Stream Processing Unit' (SPU) chain. At the OS-level, Chromium pretends that it IS the OpenGL library, hence it is transparent to the application.
Some side-notes: -This means that Gem might not need to be modified at all (!) ...and it looks like Chromium supports pretty much all of standard OpenGL through version 1.5, which should be sufficient.
- There exists a Tilesort SPU. This distributes the stream to various
machines, creating a tiled display (ie, each machine gets one tile to render). Furthermore, there is a non-planar version of Tilesort SPU where you can specify different viewing frustums for each machine - Perfect for a CAVE, or immersive environment! (see http://chromium.sourceforge.net/doc/nonplanar.html for more details).
- To answer your texture question: there's a distributed texture SPU...
from the Chromium docs: "The idea is that a program can use write-mode to distribute texture image data on a rendering cluster the first time the program is run. Then, the second and subsequent runs can use read-mode to quickly read the texture data from the render servers, instead of passing it all through the Chromium tilesort SPU."
- I didn't see any mention of dynamic textures though :( ...but if
you're talking about video files that already exist on disk somewhere, then perhaps it's possible. See http://brighton.ncsa.uiuc.edu/%7Eprajlich/wall/ppb.html for an example of showing movies on a tiled wall display using Chromium.
I'm going to look into Chromium a bit more. We'll be in touch again sometime soon.
Cheers, Mike Wozniewski
P.S. I've added one more person to the discussion: Jeremy Cooperstock, who is our project supervisor. I believe that some of you probably know each other already.
Hey.
I did not see a link to any in depth documentation from chromium.
Check http://chromium.sourceforge.net/doc/index.html. Very comprehensive.
I'm not a c++ programmer so I'm not sure what it would involve to build these functions into GL wappers for Gem. I'm not sure how the functions latch onto an existing context, and how the whole thing works architecturally (what parts run on what machines, master slave connections etc..
Well, from the docs, it seems that we don't have to do anything at all to Gem. This is because Chromium disguises itself as the OpenGL library - ie, when a GL instruction is made, Chromium intercepts and the regular system OpenGL library sits idle.
I just wonder about performance, considering the speed of the AGP bus for texture transfers vs ethernet transport between the source and destination for the texture! especially if your talking about moving video...
So according to http://brighton.ncsa.uiuc.edu/%7Eprajlich/wall/ppb.html, when distributing video, the movie has to first be played in "write mode", where all textures are cached onto the disks across the cluster. Then subsequent playback is done in "read mode", where it's just read from local disk on each machine. I see problems with this in that all videos have to exist on disk first (no streaming from live cameras), the first playback is going to be SLOW, and if you have many many video clips this could be extremely annoying.
(are you guys even using video in your cave application?)
Not yet. But we will eventually want to put video avatars of remote participants into the world (ouch - this is not going to work with the above mentioned strategy).
-Mike
Thanks for the link, I'll take a look when I have more time.
Ok so its Chromium itself that passes the texture data through the cluster. Indeed video would be more difficult, but with a 1Gbps multicast lan one should easily be able to distribute a DV stream to all machines... in theory! Are there multicast DV streamers???
The streaming part fits very well into TOT.
I've CCed Franz Hildgen and Simon Piette who are looking after the DV point-2-point application teleCHACHA.
Franz and Simon, we're talking about pd/Gem working in a cluster context, where the GL context is forwarded to a number of machines that each processes and projects one part of the image. This is very closely related to the lighTWIST and pixelTANGO integration problem.
Would it be possible to multicast a DV stream to all the cluster machines, so that each could use the stream in its portion of the final image?
Mike Wozniewski from McGill is looking at using chromium with pd/Gem for a cave application.
B>
Mike Wozniewski wrote:
Hey.
I did not see a link to any in depth documentation from chromium.
Check http://chromium.sourceforge.net/doc/index.html. Very comprehensive.
I'm not a c++ programmer so I'm not sure what it would involve to build these functions into GL wappers for Gem. I'm not sure how the functions latch onto an existing context, and how the whole thing works architecturally (what parts run on what machines, master slave connections etc..
Well, from the docs, it seems that we don't have to do anything at all to Gem. This is because Chromium disguises itself as the OpenGL library
- ie, when a GL instruction is made, Chromium intercepts and the regular
system OpenGL library sits idle.
I just wonder about performance, considering the speed of the AGP bus for texture transfers vs ethernet transport between the source and destination for the texture! especially if your talking about moving video...
So according to http://brighton.ncsa.uiuc.edu/%7Eprajlich/wall/ppb.html, when distributing video, the movie has to first be played in "write mode", where all textures are cached onto the disks across the cluster. Then subsequent playback is done in "read mode", where it's just read from local disk on each machine. I see problems with this in that all videos have to exist on disk first (no streaming from live cameras), the first playback is going to be SLOW, and if you have many many video clips this could be extremely annoying.
(are you guys even using video in your cave application?)
Not yet. But we will eventually want to put video avatars of remote participants into the world (ouch - this is not going to work with the above mentioned strategy).
-Mike
Greetings all, IIRC, the TOT project is using DVTS or a derivative thereof, in which case, the software is already multicast capable (although perhaps not in the base release).
Our UltraVideoconferencing system is expected to gain multicast capability (hopefully for both IPv4 and IPv6) later this year and this would serve (at least) for DV, analog, or SDI video distribution. Since our needs entail processing of the video as per the requirements of each receiver, and with minimal latency, DV would not be a suitable candidate for this purpose, given the encoding and decoding costs, so we're considering instead raw transmission of a bounding box region of segmented (i.e. background-removed) video. In the typical case, this payload would fit comfortably on a 100Mbps network, although segmentation problems or scene discontinuities that lead to full frame transmission would require either compression or gigabit infrastructure.
- Jeremy
B. Bogart wrote:
Thanks for the link, I'll take a look when I have more time.
Ok so its Chromium itself that passes the texture data through the cluster. Indeed video would be more difficult, but with a 1Gbps multicast lan one should easily be able to distribute a DV stream to all machines... in theory! Are there multicast DV streamers???
The streaming part fits very well into TOT.
I've CCed Franz Hildgen and Simon Piette who are looking after the DV point-2-point application teleCHACHA.
Franz and Simon, we're talking about pd/Gem working in a cluster context, where the GL context is forwarded to a number of machines that each processes and projects one part of the image. This is very closely related to the lighTWIST and pixelTANGO integration problem.
Would it be possible to multicast a DV stream to all the cluster machines, so that each could use the stream in its portion of the final image?
Mike Wozniewski from McGill is looking at using chromium with pd/Gem for a cave application.
B>
Mike Wozniewski wrote:
Hey.
I did not see a link to any in depth documentation from chromium.
Check http://chromium.sourceforge.net/doc/index.html. Very comprehensive.
I'm not a c++ programmer so I'm not sure what it would involve to build these functions into GL wappers for Gem. I'm not sure how the functions latch onto an existing context, and how the whole thing works architecturally (what parts run on what machines, master slave connections etc..
Well, from the docs, it seems that we don't have to do anything at all to Gem. This is because Chromium disguises itself as the OpenGL library
- ie, when a GL instruction is made, Chromium intercepts and the regular
system OpenGL library sits idle.
I just wonder about performance, considering the speed of the AGP bus for texture transfers vs ethernet transport between the source and destination for the texture! especially if your talking about moving video...
So according to http://brighton.ncsa.uiuc.edu/%7Eprajlich/wall/ppb.html, when distributing video, the movie has to first be played in "write mode", where all textures are cached onto the disks across the cluster. Then subsequent playback is done in "read mode", where it's just read from local disk on each machine. I see problems with this in that all videos have to exist on disk first (no streaming from live cameras), the first playback is going to be SLOW, and if you have many many video clips this could be extremely annoying.
(are you guys even using video in your cave application?)
Not yet. But we will eventually want to put video avatars of remote participants into the world (ouch - this is not going to work with the above mentioned strategy).
-Mike
Hi Jeremy,
Ah I did not realize DVTS was multicast capable.
Is UltraVideoconferencing open-source or IP-restricted? Indeed the project sounds very interesting. I hope to hear about the future developments. Have you spoken to how this may help the TOT lighTWIST system?
B.
Jeremy Cooperstock wrote:
Greetings all, IIRC, the TOT project is using DVTS or a derivative thereof, in which case, the software is already multicast capable (although perhaps not in the base release).
Our UltraVideoconferencing system is expected to gain multicast capability (hopefully for both IPv4 and IPv6) later this year and this would serve (at least) for DV, analog, or SDI video distribution. Since our needs entail processing of the video as per the requirements of each receiver, and with minimal latency, DV would not be a suitable candidate for this purpose, given the encoding and decoding costs, so we're considering instead raw transmission of a bounding box region of segmented (i.e. background-removed) video. In the typical case, this payload would fit comfortably on a 100Mbps network, although segmentation problems or scene discontinuities that lead to full frame transmission would require either compression or gigabit infrastructure.
- Jeremy
B. Bogart wrote:
Thanks for the link, I'll take a look when I have more time.
Ok so its Chromium itself that passes the texture data through the cluster. Indeed video would be more difficult, but with a 1Gbps multicast lan one should easily be able to distribute a DV stream to all machines... in theory! Are there multicast DV streamers???
The streaming part fits very well into TOT.
I've CCed Franz Hildgen and Simon Piette who are looking after the DV point-2-point application teleCHACHA.
Franz and Simon, we're talking about pd/Gem working in a cluster context, where the GL context is forwarded to a number of machines that each processes and projects one part of the image. This is very closely related to the lighTWIST and pixelTANGO integration problem.
Would it be possible to multicast a DV stream to all the cluster machines, so that each could use the stream in its portion of the final image?
Mike Wozniewski from McGill is looking at using chromium with pd/Gem for a cave application.
B>
Mike Wozniewski wrote:
Hey.
I did not see a link to any in depth documentation from chromium.
Check http://chromium.sourceforge.net/doc/index.html. Very comprehensive.
I'm not a c++ programmer so I'm not sure what it would involve to build these functions into GL wappers for Gem. I'm not sure how the functions latch onto an existing context, and how the whole thing works architecturally (what parts run on what machines, master slave connections etc..
Well, from the docs, it seems that we don't have to do anything at all to Gem. This is because Chromium disguises itself as the OpenGL library
- ie, when a GL instruction is made, Chromium intercepts and the regular
system OpenGL library sits idle.
I just wonder about performance, considering the speed of the AGP bus for texture transfers vs ethernet transport between the source and destination for the texture! especially if your talking about moving video...
So according to http://brighton.ncsa.uiuc.edu/%7Eprajlich/wall/ppb.html, when distributing video, the movie has to first be played in "write mode", where all textures are cached onto the disks across the cluster. Then subsequent playback is done in "read mode", where it's just read from local disk on each machine. I see problems with this in that all videos have to exist on disk first (no streaming from live cameras), the first playback is going to be SLOW, and if you have many many video clips this could be extremely annoying.
(are you guys even using video in your cave application?)
Not yet. But we will eventually want to put video avatars of remote participants into the world (ouch - this is not going to work with the above mentioned strategy).
-Mike
UltraVideoconferencing is closed source but the binaries are freely available for research use. I'm not up on the details of lighTWIST but if there's some relevant mutual interest, we should indeed talk.
- Jeremy
B. Bogart wrote:
Hi Jeremy,
Ah I did not realize DVTS was multicast capable.
Is UltraVideoconferencing open-source or IP-restricted? Indeed the project sounds very interesting. I hope to hear about the future developments. Have you spoken to how this may help the TOT lighTWIST system?
B.
Jeremy Cooperstock wrote:
Greetings all, IIRC, the TOT project is using DVTS or a derivative thereof, in which case, the software is already multicast capable (although perhaps not in the base release).
Our UltraVideoconferencing system is expected to gain multicast capability (hopefully for both IPv4 and IPv6) later this year and this would serve (at least) for DV, analog, or SDI video distribution. Since our needs entail processing of the video as per the requirements of each receiver, and with minimal latency, DV would not be a suitable candidate for this purpose, given the encoding and decoding costs, so we're considering instead raw transmission of a bounding box region of segmented (i.e. background-removed) video. In the typical case, this payload would fit comfortably on a 100Mbps network, although segmentation problems or scene discontinuities that lead to full frame transmission would require either compression or gigabit infrastructure.
- Jeremy
B. Bogart wrote:
Thanks for the link, I'll take a look when I have more time.
Ok so its Chromium itself that passes the texture data through the cluster. Indeed video would be more difficult, but with a 1Gbps multicast lan one should easily be able to distribute a DV stream to all machines... in theory! Are there multicast DV streamers???
The streaming part fits very well into TOT.
I've CCed Franz Hildgen and Simon Piette who are looking after the DV point-2-point application teleCHACHA.
Franz and Simon, we're talking about pd/Gem working in a cluster context, where the GL context is forwarded to a number of machines that each processes and projects one part of the image. This is very closely related to the lighTWIST and pixelTANGO integration problem.
Would it be possible to multicast a DV stream to all the cluster machines, so that each could use the stream in its portion of the final image?
Mike Wozniewski from McGill is looking at using chromium with pd/Gem for a cave application.
B>
Mike Wozniewski wrote:
Hey.
I did not see a link to any in depth documentation from chromium.
Check http://chromium.sourceforge.net/doc/index.html. Very comprehensive.
I'm not a c++ programmer so I'm not sure what it would involve to build these functions into GL wappers for Gem. I'm not sure how the functions latch onto an existing context, and how the whole thing works architecturally (what parts run on what machines, master slave connections etc..
Well, from the docs, it seems that we don't have to do anything at all to Gem. This is because Chromium disguises itself as the OpenGL library
- ie, when a GL instruction is made, Chromium intercepts and the
regular system OpenGL library sits idle.
I just wonder about performance, considering the speed of the AGP bus for texture transfers vs ethernet transport between the source and destination for the texture! especially if your talking about moving video...
So according to http://brighton.ncsa.uiuc.edu/%7Eprajlich/wall/ppb.html, when distributing video, the movie has to first be played in "write mode", where all textures are cached onto the disks across the cluster. Then subsequent playback is done in "read mode", where it's just read from local disk on each machine. I see problems with this in that all videos have to exist on disk first (no streaming from live cameras), the first playback is going to be SLOW, and if you have many many video clips this could be extremely annoying.
(are you guys even using video in your cave application?)
Not yet. But we will eventually want to put video avatars of remote participants into the world (ouch - this is not going to work with the above mentioned strategy).
-Mike
B. Bogart wrote:
Hey all,
I've moved the discussion onto the gem-dev list. I think the pders have had enough of this discussion!
just for the records and because chromium didn't work immediately for me:
how to set up pd/Gem with chromium
1. get chromium from chromium.sf.net, and build it (it seems to require gmake which i don't have; just change line 65 of config/Linux.mk to "MAKE = make -s" and go ahead)
2. set PATH & LD_LIBRARY_PATH like the docs suggest 2a export PATH=$PATH:/path/to/chromium/bin/Linux 2b export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/chromium/lib/Linux
3. IMPORTANT: somehow i additionally needed to enforce the chromium-crfaker-lib to be loaded 3a export LD_PRELOAD=/path/to/chromium/lib/Linux/libcrfaker.so
4. IMPORTANT: to make 3. work, you have to make sure that pd is _not_ setuid "root" (since for obvious security reasons, you must not trick a setuid programm to preload some libraries)
5. run chromium
6. have fun
probably i should add these to the FAQ (just for my brain: AviSynth should be mentioned there too)
mfg.asdr IOhannes
-
B. Bogart -
IOhannes m zmoelnig -
Jeremy Cooperstock -
Mike Wozniewski