hi folks,
i've been having a bit of break from the keyboard so apologies if i'm rehashing here... it looks like the yuv stuff is causing some really good discussion about GEM development. excellent!
i might hazard my opinion on some of the issues with yuv: - in general i suspect it's more costly to convert from one colour space to another to do a particular operation (say chromakey or lumakey) than it is to simply use that operation in the other colour space, even if it's a little awkward. this is even more the case if you then want to convert back to the original colour space. in the most extreme case, doing a pixel by pixel format conversion in the inner loop of the algorithm is going to be better than doing an image conversion and then running the algorithm. - as far as i can see, the only issues of platform/machine dependence with yuv are in the film codec and texture upload stages. format conversions can easily be included in the relevant objects when the underlying software and hardware do not provide the feature natively.
for these reasons (and for others more capricious), i would argue: - that all objects should be pix_XXXX objects. - all pix_ objects should operate on all image types (luminance, rgb, rgba, yuv), with the exception of the few objects which are truly format restricted (only those explicitly using the alpha channel i think) which should print an (accurate!) error message. - format conversions should only be done with explicit objects (pix_toyuv), except in image source (e.g. pix_film) and image sink (e.g. pix_texture) object where they may be done implicitly. - image sources should allow their format to be chosen through an argument or a message. they should announce the format when the image or film is loaded. should their default be system dependent to maximise performance, or system independent to maximise consistency? - a pix_info object should provide information at its outputs about the image passing through it: pixel dimensions, format and anything else relevant
it would be relatively easy to add pix_ base class automatic format conversion support as a stop-gap. this can be done with a nasty out-of-place conversion and reconversion with a performance warning printed to the console. this would allow this general behaviour to be acheived in the short term before implementing each pix_ object for each image format.
hmm i wonder if there are clever ways to use generic programming techniques to allow the algorithms to be implemented almost independently of the image format... and to allow new image formats to be added without adding code to the pix_s?
in general, i don't think that there's a really easy solution to automatic format negotiation in video data flow trees is there? i've done a bit of work with directshow on windows which tries to be very general and succeeds in being rather comlicated and hard to use...
daniel
chris clepper writes:
as ben pointed out: how about sending a "want YUV"-message to a proper [pix_film] and not having to worry whether you should need a [pix_filmBGRA_ext] or [pix_filmLUMINANCE] ?
that would work, but the question of feedback to the user remains. how do you make it clear which objects work with which color-space properly? and when do you need to convert? would this be automatic or user controlled?
the complexity really depends on the operation. for proper chroma-keying and color adjustment one must know about yuv color space. other functions like blurring, mixing and masking are just like their rgb counterparts.
so why segregating them ?
i think i've said this before:
At 6:35 PM -0600 12/16/02, chris clepper wrote:
here is my reasoning for making the [yuv_] set objects:
- a separate set of objects would be easier to leave out of compiles
for platforms that don't support it.
...
- there is not a one to one correlation between yuv and rgb color-spaces.
...
- Gem already had the pix_ naming convention which made it clear
that pix = image processing and that stringing pix_objects together would result in a functional chain. my initial attempts to extend pix to yuv seemed to destroy that, so i decided that yuv_objects would make their own chains to avoid confusion.
...
i was rather thinking of something like [pix_film] (on demand, tries to load YUV, if it fails (because the codec does not support it (is this possible), loads RGBA) -> [pix_yuv] (if pix_film returns YUV this does *nothing*, if we only have RGBA it converts to YUV) -> [pix_luma-key] (everything done in YUV-space) -> [pix_textureformat] (converts into a format that can be textured by the openGL-implementation - on macOS it might do nothing) -> [pix_texture]
ok. how about this: pix_film -> pix_luma_key -> pix_alpha -> pix_composite -> pix_texture
that's a total mess to mix and match yuv and rgba when you have alpha in one and not the other. i don't immediately see a good solution to this apart from lots of conversions that either happen in the objects automatically or a bunch of conversion objects added to the chain.
i can go on and on with examples like this, and would have to test all of these combinations if yuv_ is integrated into pix_. pix_ objects imply that they all work together, and that should be the case if yuv is added. also it seems like if conversion is going to be necessary for most situations and platforms then maybe it shouldn't be included in the CVS at all.
i will not have linux-only or macOS-only stuff in Gem as it is.
i just don't see how that's possible to maintain. osx is not linux which is not windows, so there has to be some things that can be done on one platform and not another. otherwise GEM just becomes a lowest common denominator of these platforms, and not a full featured app on any of them. if you don't take advantage of the strengths that each platform has, then why even bother making the app run on it in the first place?
cgc
--
PD-dev mailing list PD-dev@iem.kug.ac.at http://iem.kug.ac.at/cgi-bin/mailman/listinfo/pd-dev