On Sat, 27 May 2006, Miller Puckette wrote:
To begin with at least, I'm hoping to be able to do all video operations except storage using floating-point numbers, thereby re-using the usual tilde objects. The disadvantage of this approach is that, if you want to "sample" an image, to get decent cache behavior you'd want the possibility of storing it in a data-reduced way, as 8-bit integers or perhaps using YUYV packing. (for example, a 512x768 color video frame takes almost 5MB to store in floating point, but only about 0.7 MB in 8-bit YUYV.)
I sort of have the same feelings as Christian and others. If you do video processing, you probably want it to be as fast as possible, because it still takes a considerable amount of a modern CPU's power. Considering that most building blocks will be different from the ones that are used in audio processing, you will have to reimplement them anyhow, and the best would be to implement them as fast as possible.
So there will probably have to be a suite of new objects for storing 2D arrays in fixed point formats, variously trading off memory compaction with processing time needed to get the data in and out. There will probably also want to be a choice of interpolation strategies. Probably the design can look like the table/tabwrite/tabread/tabread4/tabwrite~/tabread~/tabread4~ objects.
Also like "table" objects I want to publish an API so that pdp and gem can read and write into image storage buffers.
I'll probably base the actual video I/O objects on the way Tom Shouten did it in pdp - perhaps pdp would then be able to use pd's video objects directly if I can get the design just right.
I am not sure if I understand what you mean with video I/O objects, but what I understand (output to screen and input from video sources) is the part of pdp that I think is not well designed. I find it strange having to use pdp_xv or pdp_glx as output object, depending on what hardware I have. This decision should be taken by the system, or changeable by messages, not hardcoded in a patch.
ideas and opinions welcome!
If I understand correctly, what you want to do is line by line processing for images. I am not a video specialist, by this might make some algorithms a lot harder to implement. Do you have some ideas about how a convolution with a matrix would look like, for the user and the internal implementation ?
And then, asuming that you have to use a compact representation of the image in memory, because otherwise cache behaviour will destroy performance completely, you would also have to pack and unpack the image data on every step for algorithms that are based on different frames in time.
Well, sorry if I didn't really get what it is all about, but I also try to figure out why we need yet another video extension to pd, instead of fixing the ones that exist.
Couldn't we just incorporate pdp and make it run on windows and macosx ?
Cheers, Günter
cheers Miller
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev