Hi all,
Thinking out loud is always dangerous... particularly on-list... but I thought I'd throw expand on the generic programming idea I mentioned a while ago.
chris clepper writes:
The idea of auto-conversion in each object when one object lacks the code for one color space is a bad idea. The performance hit will be quite high and at that point you might as well do software openGL rendering as well.
As far as I can see, the best outcome is to have implementations of pix objects for luminance, BGR, RGBA and YUV images. It would also be useful (in the long run) to have the potential for images with floating point components rather than integer. How can these things be achieved?
I think that the golden rules as far as performance is concerned, are: 1) use the smallest possible representation 2) minimise copying (and therefore format conversion) 3) use the representation that best suits the operations in the render chain
Difficulties are: 1) golden rules can be contradictory... need to find best balance for any given render chain on any particular hardware. This would be rather hard to do automatically - although not impossible. 2) limited support for some pixel formats from codecs and textures in openGL (on some platforms) 3) nobody wants to write code for four (or more) pixel formats 4) this is inherently complicated 5) we want to achieve: - sensible (efficient, easy) default behaviour - easy probing to see what's actually going on with image formats - easy forcing of particular behaviour (ie force the use of particular formats and/or conversions to occur).
Idea: To minimise the coding effort for multiple formats by using pixel abstraction and generic programming.
Most pix_ type operations seem to work according to one of the following schemes: - step through each pixel, process pixel, write back pixel - take a whole pixel from one part of the image, write whole pixel to another part of the image
Can the algorithms be abstracted away from the pixel format to reduce the effort of coding the image processing algorithms? They can to some extent. - If whole pixels are being moved around then the format isn't important to the algorithm if there's the notion of a generic pixel object. - Pixel by pixel conversion to/from a standard format could be done when the image (or pixel) is accessed and component level processing is done. The format would probably be RGBA. This is how some systems do pixel format abstraction I think. - Pixel processing usually uses one or more of the following values: red, blue, green, alpha channel luminance chrominance saturation, lightness (or value or ...) The values used are usually part of the algorithm... and should be chosen according to the algorithm rather than the pixel format... the pixel object class should be able to provide these values for any pixel (regardless of its underlying format). Perhaps the pix_ object can be interrogated to find its preferred pixel format.
If this is done using generic programming techniques, then the polymorphism can be collapsed at compile time, so the code can be quite efficient... and the compiler can do loop unrolling or whatever tricks it knows how to do. It would also be possible to provide optimized implementations for particular formats, if so desired, which would allow MMX/ altivec to be added.
These things might actually make it quite easy to write fast pix_ objects quickly... and get versions for each pixel format from the same code.
Is this all a load of baloney, or does anyone think it might fly? It seems to offer the possibility of implementing each pix_ object using the most suitable format and getting implementations for other formats automatically. The performance for the preferred format should be as good as a direct implementation (?) and the performance of the other formats should be reasonable (?)
I'll hack together some non-GEM based code to explore this idea. I'll let you know how it goes...
Daniel