hey,
...this weekend I wrote an object that allows offscreen rendering (or "render-to-texture") on a per renderchain basis, using framebuffer objects (FBO's, GL_EXT_framebuffer_object): it is really useful for playing with multiple rendering passes and shaders...I'd like to commit it, but it occurred to me that it might be good to see what we should name it, and where it should go in the folder hierarchy? Currently it's named [gemframebuffer], but really I think [gemoffscreen] would be better, since it could be implemented using pbuffers instead of FBO's, which would allow for some backward's compatibility (I think)...?
...also, on my harddrive I put it in src/controls (it would be inserted into a gemchain after [gemhead]), but I guess it could also go in src/manips, or even src/Nongeos?
...to give a bit more overview, it works by setting up color/depth/ stencil rgb or float buffer(s) that you can render into, and basically it get's enabled at the start of render() and disabled in the postrender() call...the buffer is also bound to a texture ID, which can then be used in later gemchains (currently via [pix_multitexture]), where it currently requires using the glwrappers to make a geo to display on: I'd like to fix this so that there's an easier way to pass textures around based on their ID's: any ideas? Unless there's a way to determine width/height from a given texture ID, I think that dimension data will also have to be passed...
james
gembuffer ? gembuf? (gemwin)
does the buffer store pixels? or gemlist info?
manips makes sense I think, or a buffer folder?
I think the buffer calling should be like the pix_buffer stuff, where you have a name [symbol] that stands for that buffer. I've not started using the pix_buffer stuff yet, but I hope you can also use a "float" as the name for looping through the buffers...
I've not looked at multitexture yet... I need more time.. I'm already taking time off work to do all the PD stuff on my plate.
I'm not getting the whole picture on how the buffers will work exactly, could you post a screenshot of an example patch?
Thanks for all the effort Jamie!
I'll get to pd 0.39 as soon as I can!!!
b.
james tittle wrote:
hey,
...this weekend I wrote an object that allows offscreen rendering (or "render-to-texture") on a per renderchain basis, using framebuffer objects (FBO's, GL_EXT_framebuffer_object): it is really useful for playing with multiple rendering passes and shaders...I'd like to commit it, but it occurred to me that it might be good to see what we should name it, and where it should go in the folder hierarchy? Currently it's named [gemframebuffer], but really I think [gemoffscreen] would be better, since it could be implemented using pbuffers instead of FBO's, which would allow for some backward's compatibility (I think)...?
...also, on my harddrive I put it in src/controls (it would be inserted into a gemchain after [gemhead]), but I guess it could also go in src/manips, or even src/Nongeos?
...to give a bit more overview, it works by setting up color/depth/ stencil rgb or float buffer(s) that you can render into, and basically it get's enabled at the start of render() and disabled in the postrender() call...the buffer is also bound to a texture ID, which can then be used in later gemchains (currently via [pix_multitexture]), where it currently requires using the glwrappers to make a geo to display on: I'd like to fix this so that there's an easier way to pass textures around based on their ID's: any ideas? Unless there's a way to determine width/height from a given texture ID, I think that dimension data will also have to be passed...
james
GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
On Nov 29, 2005, at 1:03 PM, B. Bogart wrote:
gembuffer ? gembuf? (gemwin)
does the buffer store pixels? or gemlist info?
...neither: this is a texture on the graphics card, so you can imagine it as tho it is an offscreen window that you never see unless you apply it to a primitive like any other texture...
manips makes sense I think, or a buffer folder?
...I tend toward control and manips: I don't think we need a whole new folder, because we should be able to roll in backward's compatibility via c++ magic...
I think the buffer calling should be like the pix_buffer stuff, where you have a name [symbol] that stands for that buffer. I've not started using the pix_buffer stuff yet, but I hope you can also use a "float" as the name for looping through the buffers...
...this is pretty different from pix_buffer, and works at a whole other level...however, this "named buffer" stuff is applicable to the eventual bigger idea of the multiple_window branch, where you'd be able to associate multiple chains to either offscreen or onscreen windows (or both?)...but this is kind of a first step that enables lotsa cool stuff NOW ;-)
I've not looked at multitexture yet... I need more time.. I'm already taking time off work to do all the PD stuff on my plate.
...multitexture was really part of a "trilogy" of related changes to bring GEM into the current "state of the art"...
l8r, james
B. Bogart wrote:
gembuffer ? gembuf? (gemwin)
james tittle wrote:
name it, and where it should go in the folder hierarchy? Currently it's named [gemframebuffer], but really I think [gemoffscreen] would be
as you state correctly, the name [gembuffer] would be coherent with the name [gemwindow], so the functionality should be coherent too. (imo, the same goes for [gemframebuffer] and [gemoffscreen].) i foresee nameclashes with the multiple_windows (whenever they will come...)
...also, on my harddrive I put it in src/controls (it would be inserted into a gemchain after [gemhead]), but I guess it could also go in src/manips, or even src/Nongeos?
could it be an option to make an alternative(!) [gemhead] out of this object?. so people could decide whether an entire gemchain is onscreen ([gemhead]) or offscreen ([???]). if so, i would vote for something really clumsy like [gemhead_os] ("clumsy" since i cannot think of a better name, not because the object _should_ be named clumsy of course)
mfg.ads.r IOhannes
B. Bogart wrote:
gembuffer ? gembuf? (gemwin)
james tittle wrote:
name it, and where it should go in the folder hierarchy? Currently it's named [gemframebuffer], but really I think [gemoffscreen] would be
as you state correctly, the name [gembuffer] would be coherent with the name [gemwindow], so the functionality should be coherent too. (imo, the same goes for [gemframebuffer] and [gemoffscreen].) i foresee nameclashes with the multiple_windows (whenever they will come...)
...also, on my harddrive I put it in src/controls (it would be inserted into a gemchain after [gemhead]), but I guess it could also go in src/manips, or even src/Nongeos?
could it be an option to make an alternative(!) [gemhead] out of this object?. so people could decide whether an entire gemchain is onscreen ([gemhead]) or offscreen ([???]). if so, i would vote for something really clumsy like [gemhead_os] ("clumsy" since i cannot think of a better name, not because the object _should_ be named clumsy of course)
How about putting that functionality into gemhead:
[gemhead 1] (render pri 1, onscreen) [gemhead 1 mybuffer] (pri 1, render to buffer named mybuffer)
Could be easy to test debugging by removing the one argument (or setting the message)
Actually this (ui wise) would work for multiple windows too,
[gemhead 1 mywindow] Then its transparent to the user if its a named window [gemwin 30 mywindow] or a named buffer....
Just thinking aloud.
b.
mfg.ads.r IOhannes
GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
ben,
On Nov 30, 2005, at 1:40 PM, B. Bogart wrote:
How about putting that functionality into gemhead:
[gemhead 1] (render pri 1, onscreen) [gemhead 1 mybuffer] (pri 1, render to buffer named mybuffer)
Could be easy to test debugging by removing the one argument (or setting the message)
Actually this (ui wise) would work for multiple windows too,
[gemhead 1 mywindow] Then its transparent to the user if its a named window [gemwin 30 mywindow] or a named buffer....
...this is basically the current proposal for multiple_window future development (named buffers/windows, association of gemchains to either or both, ...) Time to review my post "musings on multiple_window" and the following discussion onlist starting 2/1/2005...
Just thinking aloud.
...good to hear ;-)
james
On Nov 30, 2005, at 1:40 PM, B. Bogart wrote:
B. Bogart wrote:
gembuffer ? gembuf? (gemwin)
james tittle wrote:
name it, and where it should go in the folder hierarchy? Currently it's named [gemframebuffer], but really I think [gemoffscreen] would be
as you state correctly, the name [gembuffer] would be coherent with the name [gemwindow], so the functionality should be coherent too. (imo, the same goes for [gemframebuffer] and [gemoffscreen].) i foresee nameclashes with the multiple_windows (whenever they will come...)
...also, on my harddrive I put it in src/controls (it would be inserted into a gemchain after [gemhead]), but I guess it could also go in src/manips, or even src/Nongeos?
could it be an option to make an alternative(!) [gemhead] out of this object?. so people could decide whether an entire gemchain is onscreen ([gemhead]) or offscreen ([???]). if so, i would vote for something really clumsy like [gemhead_os] ("clumsy" since i cannot think of a better name, not because the object _should_ be named clumsy of course)
How about putting that functionality into gemhead:
[gemhead 1] (render pri 1, onscreen) [gemhead 1 mybuffer] (pri 1, render to buffer named mybuffer)
Could be easy to test debugging by removing the one argument (or setting the message)
Actually this (ui wise) would work for multiple windows too,
[gemhead 1 mywindow] Then its transparent to the user if its a named window [gemwin 30 mywindow] or a named buffer....
And argument and a inlet, maybe for the buffer name?
.hc
mfg.ads.r IOhannes
GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
________________________________________________________________________ ____
"If nature has made any one thing less susceptible than all others of exclusive property, it is the action of the thinking power called an idea, which an individual may exclusively possess as long as he keeps it to himself; but the moment it is divulged, it forces itself into the possession of everyone, and the receiver cannot dispossess himself of it." - Thomas Jefferson
On Nov 30, 2005, at 12:22 PM, IOhannes m zmoelnig wrote:
B. Bogart wrote:
gembuffer ? gembuf? (gemwin) james tittle wrote:
name it, and where it should go in the folder hierarchy? Currently it's named [gemframebuffer], but really I think [gemoffscreen] would be
as you state correctly, the name [gembuffer] would be coherent with the name [gemwindow], so the functionality should be coherent too. (imo, the same goes for [gemframebuffer] and [gemoffscreen].) i foresee nameclashes with the multiple_windows (whenever they will come...)
...[gembuffer] or [gemoffscreen] I can see being used in multiple_windows as equivalents to [gemwindow]...using a specific name for either framebuffer or pbuffer seems silly because they basically do the same thing; the difference being that one is easier to do than the other...perhaps it should be [gem_render2texture]?
...also, on my harddrive I put it in src/controls (it would be inserted into a gemchain after [gemhead]), but I guess it could also go in src/manips, or even src/Nongeos?
could it be an option to make an alternative(!) [gemhead] out of this object?. so people could decide whether an entire gemchain is onscreen ([gemhead]) or offscreen ([???]). if so, i would vote for something really clumsy like [gemhead_os] ("clumsy" since i cannot think of a better name, not because the object _should_ be named clumsy of course)
...hmm, hadn't thought about replacing the [gemhead]: I'll look into it...IIRC, I did use [gemhead] as the template originally, so...
...the bigger question I'm wrestling with is how to use it best once it's available? Somewhat the same problem with [pix_texture] and [pix_multitexture]...[pix_texture] and [gem_offscreen] both output a texture ID from their right outlet, ostensibly for use in a later render chain...currently I've used glwrappers to create the vertices to display these textureID's, but this only really works if you already know the texture size (and use TEXTURE_RECTANGLE) or if you use TEXTURE_2D and power of 2 dimensions...
...so, we currently don't have an object that displays a texture solely from a given texture ID: it might be necessary to have the textureID outlets also output the texture dimensions in a list, and then there could be another object that accepts these and produces the correct texcoords and such for passing onto a geo...?
...or is there a way to do this already that I'm not seeing?
james
Hi Jamie,
Good to hear we're thinking alike.
Could the texture ID be passed directly to a pix_tecture object via second inlet? I think this would match the current way things work, rather than getting the pix_ from pix_image we get it then from the object that sends the buffer IDs. Maybe the idea of associating each buffer with a symbol is better as its connectionless...
or to be a pix_ object... but then thats getting really complex.
Maybe the buffer stuff could be unified to a pix_buffer [type] be it a depot, a multi-image, a pbuffer and so on...
again just more talking w/ only a little thinking.
could you send a patch so I really understand the problem? (rather a screenshot of how it works now, and I can see if I can think of a nice way to name/position it...)
b.
On Nov 30, 2005, at 12:22 PM, IOhannes m zmoelnig wrote:
B. Bogart wrote:
gembuffer ? gembuf? (gemwin) james tittle wrote:
name it, and where it should go in the folder hierarchy? Currently it's named [gemframebuffer], but really I think [gemoffscreen] would be
as you state correctly, the name [gembuffer] would be coherent with the name [gemwindow], so the functionality should be coherent too. (imo, the same goes for [gemframebuffer] and [gemoffscreen].) i foresee nameclashes with the multiple_windows (whenever they will come...)
...[gembuffer] or [gemoffscreen] I can see being used in multiple_windows as equivalents to [gemwindow]...using a specific name for either framebuffer or pbuffer seems silly because they basically do the same thing; the difference being that one is easier to do than the other...perhaps it should be [gem_render2texture]?
...also, on my harddrive I put it in src/controls (it would be inserted into a gemchain after [gemhead]), but I guess it could also go in src/manips, or even src/Nongeos?
could it be an option to make an alternative(!) [gemhead] out of this object?. so people could decide whether an entire gemchain is onscreen ([gemhead]) or offscreen ([???]). if so, i would vote for something really clumsy like [gemhead_os] ("clumsy" since i cannot think of a better name, not because the object _should_ be named clumsy of course)
...hmm, hadn't thought about replacing the [gemhead]: I'll look into it...IIRC, I did use [gemhead] as the template originally, so...
...the bigger question I'm wrestling with is how to use it best once it's available? Somewhat the same problem with [pix_texture] and [pix_multitexture]...[pix_texture] and [gem_offscreen] both output a texture ID from their right outlet, ostensibly for use in a later render chain...currently I've used glwrappers to create the vertices to display these textureID's, but this only really works if you already know the texture size (and use TEXTURE_RECTANGLE) or if you use TEXTURE_2D and power of 2 dimensions...
...so, we currently don't have an object that displays a texture solely from a given texture ID: it might be necessary to have the textureID outlets also output the texture dimensions in a list, and then there could be another object that accepts these and produces the correct texcoords and such for passing onto a geo...?
...or is there a way to do this already that I'm not seeing?
james
GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
On Nov 30, 2005, at 12:22 PM, IOhannes m zmoelnig wrote:
could it be an option to make an alternative(!) [gemhead] out of this object?. so people could decide whether an entire gemchain is onscreen ([gemhead]) or offscreen ([???]). if so, i would vote for something really clumsy like [gemhead_os] ("clumsy" since i cannot think of a better name, not because the object _should_ be named clumsy of course)
...hmm, hadn't thought about replacing the [gemhead]: I'll look into it...IIRC, I did use [gemhead] as the template originally, so...
...ok, I'm really warming up to the idea of a new [gemhead], because [gemframebuffer] just sets up the buffer, then render occurs, then it textures it; and this maps well with what a [gemhead] does, without the texturing part...the limitation here is that it's just for one render chain...
...ultimately, we'd like to be able to have several renderchains go to one offscreen buffer, then be able to texture those results in a later gemchain...this suggests that the behaviour should be more like [pix_buffer] in that it's just an object on the canvas without a direct attachment to any chain...so I think one way to do this is to have [gem_offscreen 10 20], where the numbers refer to a continuum of [gemhead] priorities that would be rendered into the offscreen, then the gem_offscreen object would output a texID and dimensions (and possibly other texture-based info) from it's right outlet...in other words, any gemhead's with a rendering priority between 10 and 20 would go to the offscreen...this would require some change to gemman and an alternative gemhead object that has a postRendering() that doesn't get called until after a set of other gemchains had been rendered...
...hmm...this is getting a bit more complicated than my initial sketch of an easy way to get offscreen rendering: I think it'd be better to go ahead and get the code I do have into cvs so others can play with it rather than wait until we've hammered out all the possibilities it could touch...
...so, we currently don't have an object that displays a texture solely from a given texture ID: it might be necessary to have the textureID outlets also output the texture dimensions in a list, and then there could be another object that accepts these and produces the correct texcoords and such for passing onto a geo...?
...this is still up in the air, but seems to be leaning toward creating a kind of "texture_state" that would be a message outside of the gem_state that could be passed between render chains, and would perhaps include some of the following: format, type, dimensions, textureID...[pix_texture] could receive this via a right inlet as suggested before, but would treat this differently than the a GemState pix info, because it already has a lot of "texture setup" done...or maybe this should be an addition to the GemShape class?
hmm, jamie
This sounds pretty good to me,
Do the gemhead priorities effect what is happening in the buffer? If I offscreen gemhead 10-15, then does 10 get rendered first into the offscreen followed by 11, and so on?
If so this is great.
If not then maybe a single tag could work and be perhaps a little more friendly:
[gemhead 1] [gemhead 2 mybuffer] | | [pix_offscreen mybuffer] [pix_video] | | [pix_texture] [shader_stuff] | [square]
something like this?
Again this buffers would just me textures, so that they would probably only have pix_ objects in the chain?
I'm really REALY looking forward to a couple example patches.
b.
james tittle wrote:
On Nov 30, 2005, at 12:22 PM, IOhannes m zmoelnig wrote:
could it be an option to make an alternative(!) [gemhead] out of this object?. so people could decide whether an entire gemchain is onscreen ([gemhead]) or offscreen ([???]). if so, i would vote for something really clumsy like [gemhead_os] ("clumsy" since i cannot think of a better name, not because the object _should_ be named clumsy of course)
...hmm, hadn't thought about replacing the [gemhead]: I'll look into it...IIRC, I did use [gemhead] as the template originally, so...
...ok, I'm really warming up to the idea of a new [gemhead], because [gemframebuffer] just sets up the buffer, then render occurs, then it textures it; and this maps well with what a [gemhead] does, without the texturing part...the limitation here is that it's just for one render chain...
...ultimately, we'd like to be able to have several renderchains go to one offscreen buffer, then be able to texture those results in a later gemchain...this suggests that the behaviour should be more like [pix_buffer] in that it's just an object on the canvas without a direct attachment to any chain...so I think one way to do this is to have [gem_offscreen 10 20], where the numbers refer to a continuum of [gemhead] priorities that would be rendered into the offscreen, then the gem_offscreen object would output a texID and dimensions (and possibly other texture-based info) from it's right outlet...in other words, any gemhead's with a rendering priority between 10 and 20 would go to the offscreen...this would require some change to gemman and an alternative gemhead object that has a postRendering() that doesn't get called until after a set of other gemchains had been rendered...
...hmm...this is getting a bit more complicated than my initial sketch of an easy way to get offscreen rendering: I think it'd be better to go ahead and get the code I do have into cvs so others can play with it rather than wait until we've hammered out all the possibilities it could touch...
...so, we currently don't have an object that displays a texture solely from a given texture ID: it might be necessary to have the textureID outlets also output the texture dimensions in a list, and then there could be another object that accepts these and produces the correct texcoords and such for passing onto a geo...?
...this is still up in the air, but seems to be leaning toward creating a kind of "texture_state" that would be a message outside of the gem_state that could be passed between render chains, and would perhaps include some of the following: format, type, dimensions, textureID...[pix_texture] could receive this via a right inlet as suggested before, but would treat this differently than the a GemState pix info, because it already has a lot of "texture setup" done...or maybe this should be an addition to the GemShape class?
hmm, jamie
On Dec 3, 2005, at 11:44 AM, B. Bogart wrote:
This sounds pretty good to me,
Do the gemhead priorities effect what is happening in the buffer? If I offscreen gemhead 10-15, then does 10 get rendered first into the offscreen followed by 11, and so on?
If so this is great.
...yeh, that's the idea: [gem_offscreen] would act as a kind of book- end or wrapper around several gemchains...
If not then maybe a single tag could work and be perhaps a little more friendly:
[gemhead 1] [gemhead 2 mybuffer] | | [pix_offscreen mybuffer] [pix_video] | | [pix_texture] [shader_stuff] | [square]
something like this?
...ok, I'll try to draw out what I've got atm:
[gemhead 1] [gemhead 3] [gemhead 2] | | | [glsl_vertex] [glsl_vertex] [pix_image noise.tif] | | | [glsl_fragment] [glsl_fragment] [pix_texture] | | ________________| [glsl_program] [glsl_program] / | | / [gemhead 4] [gemframebuffer] [gemframebuffer]_/__________ | | \ | / \ | [pix_film] [pix_multitexture] [pix_multitexture] | | | [pix_texture] [pd QuadUsingGLWrappers] [pd QuadUsingGLWrappers] | [rectangle]
...so what this is supposed to show is a film that gets rendered offscreen to a rectangle while a glsl effect is added...then this offscreen is passed to another chain where it's also textured to another rectangle and has a 2nd effect applied...then finally this 2nd offscreen is textured to a rectangle for display in the onscreen window...
...cool thing is, this works right now! But, I'm still trying to cobble together some functionality in [pix_multitexture] (and possible [pix_texture] or some not-yet-created object) to more easily allow "texture state" to be passed between chains...atm it's quite hack-y and dependent on the gl wrappers...
Again this buffers would just me textures, so that they would probably only have pix_ objects in the chain?
...well, yes and no: yes, they are textures, but no, as in "not cpu- side" texture memory like we use in the pix_ objects (usually)...for the offscreen's to work within the pix-system, another object would need to be created to readback the texture info, either by glReadPixels or using pixelbuffer objects...
I'm really REALY looking forward to a couple example patches.
...hope the above can stand in for now ;-)
jamie
-
B. Bogart -
Hans-Christoph Steiner -
IOhannes m zmoelnig -
james tittle