I used to think the same thing, that floating-point computation would never be feasible on video... but Mathieu has demonstrated that, if you schedule the computations carefully, you can do blindingly fast video crunching in floating-point. Trouble with it is simply that it takes much more storage space, so you end up using a lot of memory bandwidth (a read or write to external memory costs the same as tens or hundreds of CPU cycles these days).
But anyway, I'll probably make it conditionally compilable in other formats so we can just benchmark it and find out for sure.
cheers Miller
On Sat, May 27, 2006 at 08:02:12PM +0200, Christian Klippel wrote:
hello miller,
Am Samstag, 27. Mai 2006 19:40 schrieb Miller Puckette:
To Pd developers,
[...snip...]
To begin with at least, I'm hoping to be able to do all video operations except storage using floating-point numbers, thereby re-using the usual tilde objects. The disadvantage of this approach is that, if you want to "sample" an image, to get decent cache behavior you'd want the possibility of storing it in a data-reduced way, as 8-bit integers or perhaps using YUYV packing. (for example, a 512x768 color video frame takes almost 5MB to store in floating point, but only about 0.7 MB in 8-bit YUYV.)
So there will probably have to be a suite of new objects for storing 2D arrays in fixed point formats, variously trading off memory compaction with processing time needed to get the data in and out. There will probably also want to be a choice of interpolation strategies. Probably the design can look like the table/tabwrite/tabread/tabread4/tabwrite~/tabread~/tabread4~ objects.
uhm .... are you sure about that? first, it doesnt make much sense to use floats on images. signed 16 bit integers would be _more_ than sufficient to represent a colour channel in a pixel. even film footage doesnt have that dynamic range.
using floats would make _every_ processing of video in realtime almost impossible, at least on current machines. its just too much data if you really want to do some f/x and stuff with it.....
secondly, what do you gain by _storing_ them in the native format, but _using_ them as floats? exactly nothing, instead you have the big penality of converting back and forth each time. add to that the fact that ram prices are really low, so there is no problem anymore to put 2 gigabyte or more into your computer.
instead it could pass a void* pointer to the data, that gets casted by the processing object to the format it has, and at the same time supplying an additional field, like a struct streaminfo{ width, height, current_pos, format, whatever.... } that gets evaluated then .... this would also allow for streams with different sizes to be used. imagine a stream of a 768x576 video where you add a 320x240 image ....
[...snip...]
ideas and opinions welcome!
cheers Miller
that was my 0.02 euro-cent .... ;)
greets,
chris
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev