Hi
i thought i would post an update concerning some of the things i have been working on.
- new object [pix_motionblur] which is a revised version of pix_blur with a slightly revised algo for improved performance and image quality. this also frees up pix_blur so it can be changed to an abstraction based on pix_convolve. i already committed this object.
- i have changed, but not committed, pix_texture so that the user can switch between power of 2 and rectangle textures on the fly via a 'mode' message. one question i have is whether or not rectangle textures actually work on linux or windows at the moment. the tests in pix_texture use GemMan::texture_rectangle_supported which works fine on OSX but i'm not sure how that would affect the other platforms. let me know if this looks ok for linux/windows, i don't want to break a critical object on another platform.
- i have pix_filmNEW playing in auto mode right now, but i have to agree with jamie's assessment that the new system really does nothing at all for OSX. it may solve some problems on linux, but it seems to actually have many of the same cross-platform issues; i've had to ifdef some stuff in order to get it to even compile. also, i don't think the structure is very clean or transparent since the various libs and APIs need to be taken care by platform. so doesn't it make more sense to have each platform manage the libs/APIs internally rather than do it in pix_filmNEW??
to maybe open up the communication a little bit, here are a few items i'm working on, or plan to work, on in the near future:
- an object for loading pixel shaders from a file. i only have an ati 9000 right now so i can only do support for the ati extensions.
- some sort of object that will use glCopyTexImage2D to grab part of the framebuffer and apply it directly to a texture. this would be a faster version of pix_snap and also enable some nifty feedback type effects.
- adding some sort of alpha info to the particle system for blending.
- figure out a way to easily exchange info between pix, particles and geos. part_info is a good start for applying data generated by the particle system to other elements in GEM, but what about using pix_ data as a basis for the particle system? or having the particle system manipulate vertices and texcoords? or how about rendering each frame of video as an array of triangle or spheres or even texture mapped cubes? i don't think that this means a type of system that requires the user to explicitly manage the data and do all the housekeeping like creating buffers, type and bounds checking, etc (a la jitter). instead a series of bridge objects can be constructed either in code or by abstraction to translate and guide the data around. admittedly i don't have much concrete code to offer at this point, but just wanted to bounce the idea off the others devs to get some input.
check out this for an example of particles and pixel interaction: http://www.shiffman.net/portfolio/reactive.html
cgc
Hello all,
I have used [pix_texture2] to display a 16:9 widescreen miniDV signal (with [pix_video_ds]) on my WinXP machine.
Regards, Dave S
<snip> one question i have is whether or not rectangle textures actually work on linux or windows at the moment. the tests in pix_texture use GemMan::texture_rectangle_supported which works fine on OSX but i'm not sure how that would affect the other platforms. let me know if this looks ok for linux/windows, i don't want to break a critical object on another platform. </snip> _______________________________________________ PD-dev mailing list PD-dev@iem.kug.ac.at http://iem.kug.ac.at/cgi-bin/mailman/listinfo/pd-dev
Hi all,
Looks like I have a bit of work on my plate to keep things moving in the Windows world!
From: "chris clepper" cclepper@artic.edu
- i have changed, but not committed, pix_texture so that the user can
switch between power of 2 and rectangle textures on the fly via a 'mode' message. one question i have is whether or not rectangle textures actually work on linux or windows at the moment. the tests in pix_texture use GemMan::texture_rectangle_supported which works fine on OSX but i'm not sure how that would affect the other platforms. let me know if this looks ok for linux/windows, i don't want to break a critical object on another platform.
This sounds good. I'd also like to implement texture naming to allow loaded textures to be shared easily - good for particles and all sorts of things.
- i have pix_filmNEW playing in auto mode right now, but i have to
agree with jamie's assessment that the new system really does nothing at all for OSX. it may solve some problems on linux, but it seems to actually have many of the same cross-platform issues; i've had to ifdef some stuff in order to get it to even compile. also, i don't think the structure is very clean or transparent since the various libs and APIs need to be taken care by platform. so doesn't it make more sense to have each platform manage the libs/APIs internally rather than do it in pix_filmNEW??
Hmm.... I think it should be possible to do things in the modular way that pix_filmNEW attempts and without making the implementation and maintenance harder for the OS/lib module than it would be for a separate object in the old pix_film system. Given the advantages of this and the fact that most of the libraries can in fact be used on most platforms this seems to make the effort worthwhile to me...?
On this subject - have the Mac folks managed to convince QT to play mpegs and flash files yet? If so it's probably time for me to do the QTML port so we can get QT playback on Windows.
to maybe open up the communication a little bit, here are a few items i'm working on, or plan to work, on in the near future:
- an object for loading pixel shaders from a file. i only have an
ati 9000 right now so i can only do support for the ati extensions.
excellent - sounds very promising. any thoughts about how to move the shader program into the patching domain?
- some sort of object that will use glCopyTexImage2D to grab part of
the framebuffer and apply it directly to a texture. this would be a faster version of pix_snap and also enable some nifty feedback type effects.
yup - as promised I'll put my version of this into CVS. I was waiting until I'd had a chance to get the performance of this up to scratch on OS X and linux but there are other people here better equipped to do this. Look for [pix_snap2tex] later tonight (my time).
- figure out a way to easily exchange info between pix, particles and
geos. part_info is a good start for applying data generated by the particle system to other elements in GEM, but what about using pix_ data as a basis for the particle system? or having the particle system manipulate vertices and texcoords? or how about rendering each frame of video as an array of triangle or spheres or even texture mapped cubes? i don't think that this means a type of system that requires the user to explicitly manage the data and do all the housekeeping like creating buffers, type and bounds checking, etc (a la jitter). instead a series of bridge objects can be constructed either in code or by abstraction to translate and guide the data around. admittedly i don't have much concrete code to offer at this point, but just wanted to bounce the idea off the others devs to get some input.
Yup - I've done some similar things in patches and externals. Easy to do the hard way with [counter]s and [pix_data] but slow. So I then went all the way to a monolithic external with lots of messages for control.
The particle system style approach is definitely more flexible however. It would be good to have geometric particle generators which could also be parameterised on lists, tables or pix_ data. E.g. generate grids of particles and have the color/ size/ displacement of each etc based on the sampling the data.
For good performance, it's probably necessary to use display lists for the particles, I imagine. Altho there's quite a lot of code in GEM devoted to caching branches in display lists they don't seem to be used often/at all - is that right?
ideas for particle system generators: - halftone screens: - grid with variable angle. - concentric circular screen - (animated) paths - circles loops, spline curves - geometric objects ... - 1-d to 3-d density function based on input list/table/pix_data - openGL feedback based system where you can define the generator as a GEM render chain using normal GEM / openGL primitives which is run in feedback mode to generate particle positions. Imagine being able to have a mesh turn into a swarming mass of particles...
other ideas: Of course it would be nice it were possible to interpolate smoothly between the different particle system generation functions somehow... so that the particles themselves move consistently.
Daniel
hi.
Zitiere chris clepper cclepper@artic.edu:
Hi
- new object [pix_motionblur] which is a revised version of pix_blur
i have noticed this, great.
quality. this also frees up pix_blur so it can be changed to an abstraction based on pix_convolve.
which reminds me, we have to make ánd abstraction-folder.
- i have changed, but not committed, pix_texture so that the user can
switch between power of 2 and rectangle textures on the fly via a 'mode' message. one question i have is whether or not rectangle textures actually work on linux or windows at the moment. the tests
rectangle textures work under linux if you have an nvidia-card, recent drivers and are compiling agains the GL-include files provided by nvidia. but there is still some bug in the [pix_texture] object, which refuses to display rectangle-textures on nvidia-cards if they are not compiled against nvidia-sources.
in pix_texture use GemMan::texture_rectangle_supported which works fine on OSX but i'm not sure how that would affect the other platforms. let me know if this looks ok for linux/windows, i don't want to break a critical object on another platform.
works great. (but of course, the underlying check for rectangle-support has to be adapted to meet (say) nvidia's extensions)
- i have pix_filmNEW playing in auto mode right now, but i have to
great (thanks a lot!)
agree with jamie's assessment that the new system really does nothing at all for OSX.
which might be true. however is there no chance that there might be other decoder-libs under mac some day ?
it may solve some problems on linux, but it seems to
it does ;-)
actually have many of the same cross-platform issues; i've had to ifdef some stuff in order to get it to even compile. also, i don't think the structure is very clean or transparent since the various libs and APIs need to be taken care by platform. so doesn't it make more sense to have each platform manage the libs/APIs internally rather than do it in pix_filmNEW??
actually that was the idea. maybe it is just really bad "design" what i have done. i (hope that i) am free for any suggestions. (but of course don't want to break your work again)
to maybe open up the communication a little bit, here are a few items i'm working on, or plan to work, on in the near future:
- an object for loading pixel shaders from a file. i only have an
ati 9000 right now so i can only do support for the ati extensions.
this really sounds cool
- some sort of object that will use glCopyTexImage2D to grab part of
the framebuffer and apply it directly to a texture. this would be a faster version of pix_snap and also enable some nifty feedback type effects.
long on my todo list (since daniel talked about this) - but never had the time.
- adding some sort of alpha info to the particle system for blending.
i thought the new libparticle already had alpha-support (the colors are now rgba instead of rgb) so this might be solved already.
- figure out a way to easily exchange info between pix, particles and
geos. part_info is a good start for applying data generated by the particle system to other elements in GEM, but what about using pix_ data as a basis for the particle system? or having the particle
.... sounds great!
have a go (don't curse too often) ;-) i'm looking forward to the next time i have access to the cvs.
mfg.as.r IOhannes
quality. this also frees up pix_blur so it can be changed to an abstraction based on pix_convolve.
which reminds me, we have to make ánd abstraction-folder.
how do you want to include this in the GEM distribution? the abstractions have to be in the search path, so do we just write a script to copy them into the extra folder or maybe alias them?
in pix_texture use GemMan::texture_rectangle_supported which works fine on OSX but i'm not sure how that would affect the other platforms. let me know if this looks ok for linux/windows, i don't want to break a critical object on another platform.
works great. (but of course, the underlying check for rectangle-support has to be adapted to meet (say) nvidia's extensions)
ok, i will commit my changes, and hopefully it will work on linux.
- i have pix_filmNEW playing in auto mode right now, but i have to
great (thanks a lot!)
agree with jamie's assessment that the new system really does nothing at all for OSX.
which might be true. however is there no chance that there might be other decoder-libs under mac some day ?
i can't say that there won't be, but quicktime covers a whole lot it's easily extend by adding codec components. for a 'basic' film player it has everything covered.
it may solve some problems on linux, but it seems to
it does ;-)
actually have many of the same cross-platform issues; i've had to ifdef some stuff in order to get it to even compile. also, i don't think the structure is very clean or transparent since the various libs and APIs need to be taken care by platform. so doesn't it make more sense to have each platform manage the libs/APIs internally rather than do it in pix_filmNEW??
actually that was the idea. maybe it is just really bad "design" what i have done. i (hope that i) am free for any suggestions. (but of course don't want to break your work again)
maybe i'm just resisting the new system because jamie and i did a lot of work to get the qt playback working well in the old system and now it requires a rewrite for the new one. i don't really understand why pix_filmNEW has to handle all of the libs and APIs for every platform. can there be a way to have a class that handles all of the libs needed for a platform and then use pix_filmNEW as the public class? i guess this is more like the 'old way' with pix_filmLinux/Darwin/NT, but with the individual libs exposed like in the new system. is it possible to really reuse the various libs on other platforms? libmpeg doesn't work on OSX just like the Mac quicktime doesn't work in linux. maybe ffmpeg is a possibility to use cross platform, but i haven't looked into it.
also, i forsee some problems with pix_videoNEW as well, but more on that after i start on it...
- adding some sort of alpha info to the particle system for blending.
i thought the new libparticle already had alpha-support (the colors are now rgba instead of rgb) so this might be solved already.
oh ok. i have version 111 in my local, but the CVS one is 120. i'll update and check it out.
cgc
hi all.
just commited one or two changes to the cvs. 1. Pixes/pix_texture now has automatic yuv2rgb conversion (on non-mac's -- i couldn't get my nvidia to texture yuv422-maps correctly) 2. Base/GemBase removed the "topmost"-warning for mac/linux (it somehow disturbed me;-)) more important: i have ifdef'ed the openGL-extensions-queries. if gem was compiled without texture_rectangle-support (for instance, because the nvidia-GL-headers haven't been installed) but was run with drivers that supported it (for instance nvidia-drivers) textures became very black. 3. Copied yuv/[yuv_emboss] to Pixes/[pix_emboss] (to have one true yuv-effect for testing) however, that was what i meant in one of my previous mails: yuv2pix is not yet finished since there are a lot of yuv-effects... 4. videoV4L: hope i have the colour-spaces right now 5. Base/GemPixUtil : added some more (integer!) colour-conversions and some other utilities like "setCsizeByFormat" (since the csize is bound to the format) and a string2format conversion ("grey" to GL_LUMINANCE). these are of course very small changes. added some documentation of the code too.
i hope, i haven't broken any things. after "cvs up; cvs ci" it compiles on my machine ;-)
chris clepper wrote:
which reminds me, we have to make ánd abstraction-folder.
how do you want to include this in the GEM distribution? the abstractions have to be in the search path, so do we just write a script to copy them into the extra folder or maybe alias them?
well, i have thought of putting them into Gem/abs/ (besides Gem/src/) ;-) this brings us back to a very non-Gem specific problem of where to put abstractions in pd-runtime. under linux there is a "make install" which installs the Gem.pd_linux and the documentation. So there is already a script which has to be extended. Similar under win(tm). How do you handle installation ? Basically i would say: if the user has to copy the binary Gem.[pd_linux/pd_darwin/dll] by hand, s/he can copy the abstractions by hand to wherever s/he likes too. If there is an installer (there should be), this will take care of it.
personally i prefer separate directories for such things (that's why there is the doc/5.references/Gem directory) - although it maybe favours name-clashes. But while the prepend of "Gem/" to the help-symbol is no problem, a Gemabs path couldn't be made totally user-transparent. <snip> But: if pd would enable a global pdrc-file this could be easily patched </snip>
ok, i will commit my changes, and hopefully it will work on linux.
i haven't had any problems yet
maybe i'm just resisting the new system because jamie and i did a lot of work to get the qt playback working well in the old system and now it requires a rewrite for the new one. i don't really understand why pix_filmNEW has to handle all of the libs and APIs for every platform. can there be a way to have a class that handles all of the libs needed for a platform and then use pix_filmNEW as the public class? i guess this is more like the 'old way' with pix_filmLinux/Darwin/NT, but with the individual libs exposed like in the new system. is it possible to really reuse the various libs on other platforms? libmpeg doesn't work on OSX just like the Mac quicktime doesn't work in linux. maybe ffmpeg is a possibility to use cross platform, but i haven't looked into it
hmm. i thought pix_filmNEW doesn't handle all the APIs from the various libs. this is done in film*. So i guess your question is: Why are there calls for both filmDarwin and filmMPEG1 in the pix_filmNEW ? if you wanted to open a special format, the first film*-class is tried. if it fails to open the file (either because the format is not supported or the library isn't present on the OS) the next one is tried. I thought it would be the same if a certain (existing) lib couldn't open a file or the lib wasn't present at all (either because of the OS or because nobody has installed it) i don't think that this is a speed issue, since this is only one (or several) void-call(s) when opening a file. .
also, i forsee some problems with pix_videoNEW as well, but more on that after i start on it...
i am interested in this
mfg.as.dr IOhannes
On Sun, 11 May 2003, chris clepper wrote:
i don't think that this means a type of system that requires the user to explicitly manage the data and do all the housekeeping like creating buffers, type and bounds checking, etc (a la jitter).
Well, Jitter does it the GridFlow way, which is already pretty abstracted out from low-level considerations. The most difficult part is actually mapping the dimensions and indices of one data structure to another in a _meaningful_ way (or at least an interesting way). The most lowlevel is direct redimensioning, but actually there are much more sophisticated transforms than mere reinterpretation of data.
In any case, a type conversion is not more that just saying that you want a type conversion to happen ([@cast uint8]), and a redim is not more than saying you want it to happen ([@redim {240 320 3}]), and all objects do check bounds automatically. So I'm not sure what kind of "housekeeping like creating buffers, type and bounds checking, etc" that you are talking about.
instead a series of bridge objects can be constructed either in code or by abstraction to translate and guide the data around. [...] just wanted to bounce the idea off the others devs to get some input.
could you please elaborate on this?
________________________________________________________________ Mathieu Bouchard http://artengine.ca/matju
Well, Jitter does it the GridFlow way, which is already pretty abstracted out from low-level considerations. The most difficult part is actually mapping the dimensions and indices of one data structure to another in a _meaningful_ way (or at least an interesting way). The most lowlevel is direct redimensioning, but actually there are much more sophisticated transforms than mere reinterpretation of data.
the 'meaningful way' is what i'm talking about. how to make it simple for the user to get some element of the pixel data, like the Y for example, and map it to another system like texcoords or vertices.
In any case, a type conversion is not more that just saying that you want a type conversion to happen ([@cast uint8]), and a redim is not more than saying you want it to happen ([@redim {240 320 3}]), and all objects do check bounds automatically. So I'm not sure what kind of "housekeeping like creating buffers, type and bounds checking, etc" that you are talking about.
ok, i'm not making any sort of value judgement about GridFLow at all, but i think you actually made the point for me: i don't think users need to care or even know about which C data type the pixel info is in and what it needs to be cast to to work in GL. there are lots of people i know and work with that really couldn't be bothered with the process of explicitly doing the buffer parsing to get at the data they want, casts and resizes - that's the sort of thing the programmers of the system are supposed to do for them. ;)
instead a series of bridge objects can be constructed either in code or by abstraction to translate and guide the data around. [...] just wanted to bounce the idea off the others devs to get some input.
could you please elaborate on this?
i do not have anything working at the moment, it was only an idea i thought the other devs might want to think about too. but basically the concept is to have objects that took info out of the pixel buffer and turned that info into something that the particle system could use or could be vertices used to built a 3D Geo, etc. the object would simply require the user to select which part of the data they want to use and optionally resize and rearrange it. a hypothetical object could be [pix_to_vertex R 64 64] which would take the red channel of the video and scale it to 64 by 64 array of vertices. of course this is quite limited in a way but also direct, and hopefully efficient to implement in C code as well. again this is purely off the top of my head with no actual system in place, so who knows what the final result will be...
cgc
On Sun, 11 May 2003, chris clepper wrote:
the 'meaningful way' is what i'm talking about. how to make it simple for the user to get some element of the pixel data, like the Y for example, and map it to another system like texcoords or vertices.
Well, the problem with that is that there is no standard way of doing so.
In video you can make a set of stereotyped effects with a few user-tunable parameters and hand that to the user. Same with each other subsystem, I guess.
When it comes to moving data across subsystems, however, I think the possibilities have not been explored enough yet, so there wouldn't be any kind of agreement on what is a good way to convert from one subsystem to another, and that is bound to evolve very much in the next years.
For my own practice I want to leave all doors open, and basically do all of the mathematical mapping myself. What I don't want is dealing with C++ and buffer overflows and writing the minute details of everything.
It's true however that leaving all doors open means someone may get lost figuring out his path. Then abstractions may represent certain forms of beaten path. But to make abstractions (bottom-up design), I'd rather have evidence that the pattern I am abstracting out is really a pattern, which is not too obvious to me at first sight, lacking long-term experience doing intermedia conversion.
ok, i'm not making any sort of value judgement about GridFLow at all, but i think you actually made the point for me: i don't think users need to care or even know about which C data type the pixel info is in and what it needs to be cast to to work in GL.
Well I expect the C datatype to become optional... I just have too many questions left unanswered regarding that, so I'm leaving most implicit conversions for later.
So would you please concentrate on the dimensionality aspect of data, which is more fundamental. Like, a 3D N-sided polygon is represented by a N-row, 3-column grid (Dim[N,3]), while a picture is represented by a Height-by-Width-by-Channels grid (Dim[Y,X,C]).
So, is the user supposed to be aware of the grid dimensions like N*3 vs Y*X*C ?
there are lots of people i know and work with that really couldn't be bothered with the process of explicitly doing the buffer parsing to get at the data they want, casts and resizes - that's the sort of thing the programmers of the system are supposed to do for them. ;)
oh. well maybe that lots of people you know really couldn't be bothered with the idea of making those intermedia mappings in the first place.
Or if they really want to do it, but can't be bothered with the dimensionality and interpretation of the data, well that begins to smell like money. Especially because it smells like hard work. I mean reifying the user's desires may require quite a bit of psychoanalysis (grin) sometimes, and then a bunch of trial-and-error.
Else they may be happy with a bunch of standard intermedia effects (whenever that comes to exist) but that's the same problem as with anything else. Not being above the tools being used, there's the risk of looking like anyone else. Artistically, there's definitely a PhotoShop style, an AdobeIllustrator/CorelDraw style, and so on. Of course a bit of that is each medium itself by its conceptual nature, but the most obvious is the stereotyping of effects.
a hypothetical object could be [pix_to_vertex R 64 64] which would take the red channel of the video and scale it to 64 by 64 array of vertices.
oh ok.
reminds me I have to implement [@slice] someday because right now taking a slice (cropping) data is done using the very flexible [@for]/[@store], but it deserves to be faster...
Of course [@slice] would work for any data type representable as grid...
________________________________________________________________ Mathieu Bouchard http://artengine.ca/matju
-
chris clepper -
Daniel Heckenberg -
David Sabine -
IOhannes zmoelnig -
Mathieu Bouchard -
zmoelnig＠iem.at