hi,
an open letter to the gem devs :)
i'm not 100% there yet but time has come to start thinking about a direct gem<->pdp/pf plug.
my plans are: * pix <-> pdp image packet converters * plug 3dp into the gem 3d render chain
the pixbufs should not pose too many problems.
plugging in the render chain seems relatively easy too. all relevant 3dp code is pure opengl. 3dp objects work like this: accept context packet, draw to or otherwize modify the context, and pass along. the context packet is not really used atm, and serves as a 'bang': the objects draw to the current gl context.
so, i'm fishing for advice.
cheers tom
Tom
The pix stuff is probably not too difficult as it will just be another blitter loop to go from one image buffer type to the other. There is already an object or two that does this correct?
The GL stuff is a bit trickier. GEM currently does everything in immediate mode or using display lists, and none of the actual vertex rendering information is passed between objects. I suppose you could insert other types of viewport, lighting and texture manipulation objects into the chain, but GEM has all of the basics covered. I have recently added some vertex array objects to CVS which do in fact pass raw vertex, color, texcoord and normals between objects much like the pixel processing objects. All of the processing is done by the CPU, but this arrangement makes for very nice and flexible possibilities (and is actually much faster than immediate mode). I would suggest finding a way to integrate your 3D objects into that rather than spend a lot of time with the somewhat obsolete Geos and so on.
Another possibility is the creation of display lists out of user defined chains of objects. This would be more flexible and faster than immediate mode, but really only useful for static data like models, and collections of basic shapes (rectangle, cube, triangles). It would interface very nicely with Cyrille's pmpd package and help performance immensely.
cgc
On Sep 7, 2004, at 1:32 PM, Tom Schouten wrote:
hi,
an open letter to the gem devs :)
i'm not 100% there yet but time has come to start thinking about a direct gem<->pdp/pf plug.
my plans are:
- pix <-> pdp image packet converters
- plug 3dp into the gem 3d render chain
the pixbufs should not pose too many problems.
plugging in the render chain seems relatively easy too. all relevant 3dp code is pure opengl. 3dp objects work like this: accept context packet, draw to or otherwize modify the context, and pass along. the context packet is not really used atm, and serves as a 'bang': the objects draw to the current gl context.
so, i'm fishing for advice.
cheers tom
PD-dev mailing list PD-dev@iem.at http://iem.at/cgi-bin/mailman/listinfo/pd-dev
On Wed, Sep 08, 2004 at 10:31:21AM -0500, chris clepper wrote:
hi chris,
The pix stuff is probably not too difficult as it will just be another blitter loop to go from one image buffer type to the other. There is already an object or two that does this correct?
i'm not sure which object you mean here.
The GL stuff is a bit trickier. GEM currently does everything in immediate mode or using display lists, and none of the actual vertex rendering information is passed between objects.
what exactly is passed from one gem object to the next, looking from the outside (i.e. pd messages)
suppose i do this: just echo the render message and pass it on, and then execute some direct GL code before passing it:
[gem...] | [3dp ...] | [gem...]
so if the 3dp object would echo anything that comes in, but use the message just as a synchronization for when to execute the GL code, would this work?
one thing i don't understand is wether gem 'compiles' a pd network into some other structure where a sequence of functions are executed, or if the drawing happens at the moment the 'context' comes in the left inlet.
I suppose you could insert other types of viewport, lighting and texture manipulation objects into the chain, but GEM has all of the basics covered.
yes. this is supposed to be an addition, not a replacement.
the thing i'm most interested in, is to keep one of the two (gem or pdp) as the master 'sequencer', and plug in objects from the other in the render chain. doing this with gem as the master seems easier atm. i don't think it will be easy for 3dp objects to feed the necessary context to gem objects.
I have recently added some vertex array objects to CVS which do in fact pass raw vertex, color, texcoord and normals between objects much like the pixel processing objects. All of the processing is done by the CPU, but this arrangement makes for very nice and flexible possibilities (and is actually much faster than immediate mode). I would suggest finding a way to integrate your 3D objects into that rather than spend a lot of time with the somewhat obsolete Geos and so on.
ok. this is a lot different than i assumed. a lot of ideas to steal :) i have a matrix object to store most of these data objects so in principle there should not be a problem. though i need to do more code reading..
Another possibility is the creation of display lists out of user defined chains of objects. This would be more flexible and faster than immediate mode, but really only useful for static data like models, and collections of basic shapes (rectangle, cube, triangles). It would interface very nicely with Cyrille's pmpd package and help performance immensely.
this could be interesting. i can 'compile' to display list in pf, and they are represented using a wrapped gl datatype, so this is another level that could be plugged. thanks for the hint.
this looks very doable. just need to solve them practical problems..
so, where do i start? gem 0.90, cvs, some branch? i tried gem cvs yesterday but the default branch did not compile..
On Sep 8, 2004, at 2:51 PM, Tom Schouten wrote:
i'm not sure which object you mean here.
...I'd guess pdp2gem/gem2pdp, which have worked in the past...they're slow, due to the "copy planar YUV to RGB or vice versa" loops...something I've been meaning to look into would be adding support for just planar YUV to YUV 4:2:2, or also allowing straight planar YUV copies (there's recently been a planar YUV quicktime codec released 3rd party, IIRC)...
The GL stuff is a bit trickier. GEM currently does everything in immediate mode or using display lists, and none of the actual vertex rendering information is passed between objects.
what exactly is passed from one gem object to the next, looking from the outside (i.e. pd messages)
...GemState is passed, which includes flags for dirty states, displaylists, lighting, smoothing, texture mapping, etc...and of course now vertex array versions of the similar...
suppose i do this: just echo the render message and pass it on, and then execute some direct GL code before passing it:
[gem...] | [3dp ...] | [gem...]
so if the 3dp object would echo anything that comes in, but use the message just as a synchronization for when to execute the GL code, would this work?
...seems like a place to start!
one thing i don't understand is wether gem 'compiles' a pd network into some other structure where a sequence of functions are executed, or if the drawing happens at the moment the 'context' comes in the left inlet.
...I don't think it's that complex, yet :-)
I suppose you could insert other types of viewport, lighting and texture manipulation objects into the chain, but GEM has all of the basics covered.
yes. this is supposed to be an addition, not a replacement.
the thing i'm most interested in, is to keep one of the two (gem or pdp) as the master 'sequencer', and plug in objects from the other in the render chain. doing this with gem as the master seems easier atm. i don't think it will be easy for 3dp objects to feed the necessary context to gem objects.
...the gemhead basically allows for sequencing the order of gl operations: don't forget that each gemhead can have a different "priority" via [set $1<, with 1 being highest priority and default being 50...so it sends out a render GemState for every [frame $1< slice (ie. fps)...I'd think the important files to check out are GemState.h and GemMan.h/GemMan.cpp...
so, where do i start? gem 0.90, cvs, some branch? i tried gem cvs yesterday but the default branch did not compile..
...what platform were you trying? CVS head is slightly garbled atm, because of an accidental (yet unfortunately incomplete) commit of some vertex array stuff...I'd try again with the "vertex_array" tag, tho it also has it's problems (osx project files aren't up-to-date plus it was merged from a not-quite-up-to-date HEAD)...
hop on board! I know I'm following what you're doing with pf/pdp/whatever!
james
On Wed, Sep 08, 2004 at 03:40:49PM -0400, James Tittle II wrote:
On Sep 8, 2004, at 2:51 PM, Tom Schouten wrote:
i'm not sure which object you mean here.
...I'd guess pdp2gem/gem2pdp, which have worked in the past...
ok. of course. i was thinking about an internal gem object. i plan to use yves' object as a template because it already solves most gem<->pdp interface and build problems.
they're slow, due to the "copy planar YUV to RGB or vice versa" loops...
yes. and the gem2pdp bridge uses glReadPixels, which is i think slow for almost all video cards/drivers..
something I've been meaning to look into would be adding support for just planar YUV to YUV 4:2:2, or also allowing straight planar YUV copies (there's recently been a planar YUV quicktime codec released 3rd party, IIRC)...
you mean as a pix buffer, or a texture?
what exactly is passed from one gem object to the next, looking from the outside (i.e. pd messages)
...GemState is passed, which includes flags for dirty states, displaylists, lighting, smoothing, texture mapping, etc...and of course now vertex array versions of the similar...
so, basicly, if i wrap the gemstate object, i should be able to get to everything. i think this might be what i'm looking for.
suppose i do this: just echo the render message and pass it on, and then execute some direct GL code before passing it:
[gem...] | [3dp ...] | [gem...]
so if the 3dp object would echo anything that comes in, but use the message just as a synchronization for when to execute the GL code, would this work?
...seems like a place to start!
perfect.
one thing i don't understand is wether gem 'compiles' a pd network into some other structure where a sequence of functions are executed, or if the drawing happens at the moment the 'context' comes in the left inlet.
...I don't think it's that complex, yet :-)
ok.
I suppose you could insert other types of viewport, lighting and texture manipulation objects into the chain, but GEM has all of the basics covered.
yes. this is supposed to be an addition, not a replacement.
the thing i'm most interested in, is to keep one of the two (gem or pdp) as the master 'sequencer', and plug in objects from the other in the render chain. doing this with gem as the master seems easier atm. i don't think it will be easy for 3dp objects to feed the necessary context to gem objects.
...the gemhead basically allows for sequencing the order of gl operations: don't forget that each gemhead can have a different "priority" via [set $1<, with 1 being highest priority and default being 50...so it sends out a render GemState for every [frame $1< slice (ie. fps)...I'd think the important files to check out are GemState.h and GemMan.h/GemMan.cpp...
thanks. this should keep me budy for a while.
so, where do i start? gem 0.90, cvs, some branch? i tried gem cvs yesterday but the default branch did not compile..
...what platform were you trying?
linux.
CVS head is slightly garbled atm, because of an accidental (yet unfortunately incomplete) commit of some vertex array stuff...I'd try again with the "vertex_array" tag, tho it also has it's problems (osx project files aren't up-to-date plus it was merged from a not-quite-up-to-date HEAD)...
hmm. maybe it's a better idea if i use the stable version of GEM, (which is 0.90, right?)
so it can be tested by more people. i don't think i can get this right and forget about it in a couple of hours, so it would be best to aim at easy testability instead of hot features for now.
hop on board! I know I'm following what you're doing with pf/pdp/whatever!
thanks :)
tom
On Sep 8, 2004, at 3:08 PM, Tom Schouten wrote:
ok. of course. i was thinking about an internal gem object. i plan to use yves' object as a template because it already solves most gem<->pdp interface and build problems.
Yes, I was referring to those existing bridge objects from Yves.
yes. and the gem2pdp bridge uses glReadPixels, which is i think slow for almost all video cards/drivers..
glReadPixels has gotten better on the Mac recently, but it is still at the mercy of the AGP design which is optimized for uploads to the card and not reads from it. Even AGP 8X still only has a 1X 233 MB/sec read rate - and that defers to any and all uploads.
something I've been meaning to look into would be adding support for just planar YUV to YUV 4:2:2, or also allowing straight planar YUV copies (there's recently been a planar YUV quicktime codec released 3rd party, IIRC)...
you mean as a pix buffer, or a texture?
I would suggest having the pix_ buffer in RAM as one very easy set of bridges to make, since those are fairly easily done. Any and all conversions have a pretty decent chance of hot-rodding through Altivec/SSE.
The texture part is quite another story as it could tie into the discussion about multiple render targets. Having GEM and PDP share contexts and textures then one could render part of the chain in one and switch it to the other for further processing.
so, basicly, if i wrap the gemstate object, i should be able to get to everything. i think this might be what i'm looking for.
Yep. Adding the GemState to all of the PDP 3D objects would pretty much allow any objects in PDP to work in a GEM render chain.
so if the 3dp object would echo anything that comes in, but use the message just as a synchronization for when to execute the GL code, would this work?
It would be interesting to actually have the PDP objects manipulate the data passed with the GemState. For example, you could write some texture coordinate manipulations, add a display list or detect lighting changes. At some point the vertex arrays will stabilize and that will offer more fun as well.
hmm. maybe it's a better idea if i use the stable version of GEM, (which is 0.90, right?)
I believe v_090 is the tag to check out.
so it can be tested by more people. i don't think i can get this right and forget about it in a couple of hours, so it would be best to aim at easy testability instead of hot features for now.
0.90 should provide you with a stable base to work from, and right now the only major changes I know about to GemState is the addition of the vertex_array data. That addition is only a few flags and four pointers, which do not change the way any of the other state data works.
Make sure you are on the gem-dev list as well!
cgc
so, basicly, if i wrap the gemstate object, i should be able to get to everything. i think this might be what i'm looking for.
Yep. Adding the GemState to all of the PDP 3D objects would pretty much allow any objects in PDP to work in a GEM render chain.
ok, then i think it's best i do it like that. still need to figure out how exactly, but it is certainly possible without too much work. actually, i can see the gem state as an extension to the already present internal GL state and work on that.
so if the 3dp object would echo anything that comes in, but use the message just as a synchronization for when to execute the GL code, would this work?
It would be interesting to actually have the PDP objects manipulate the data passed with the GemState. For example, you could write some texture coordinate manipulations, add a display list or detect lighting changes. At some point the vertex arrays will stabilize and that will offer more fun as well.
this is starting to look really good.
Make sure you are on the gem-dev list as well!
see you on gem-dev
James Tittle II wrote:
On Sep 8, 2004, at 2:51 PM, Tom Schouten wrote:
i'm not sure which object you mean here.
so if the 3dp object would echo anything that comes in, but use the message just as a synchronization for when to execute the GL code, would this work?
...seems like a place to start!
right, this should work. i was thinking of addint a "bang" message to the GEMgl...objects to call the render-routine to make them more general and not depending on the GemState (which most of them do not need at all, and the ones that would need it, do not recognize it :-()
one thing i don't understand is wether gem 'compiles' a pd network into some other structure where a sequence of functions are executed, or if the drawing happens at the moment the 'context' comes in the left inlet.
...I don't think it's that complex, yet :-)
actually it used to be that complex. however, since gem-0.87 (i think) i have stopped this, since the overhead we get from using pd's message-system for the rendering is not that big (performance loss is "only" about 50% when using GEMgl...stuff, but decreases with more complex objects).
i have no real plans to change this, as it makes things more complicated than they should be.
hop on board! I know I'm following what you're doing with pf/pdp/whatever!
right. i am really glad about all that stuff. tom, i have added you to the developers at pd-gem.sf.net (if you don't mind)
looking forward to the convention to see (most of) you in real-life very much.
mfg.a.sdr IOhannes
PS: sorry for not replying sooner to this thread, but i was a bit overworked.
-
chris clepper -
IOhannes m zmoelnig -
James Tittle II -
Tom Schouten