- Usability and high level objects:
Pro:
- user has a single object that does what he wants, using an intuitive
name
- can be optimized for speed, because its less general
the first is key because it provides readily accessible and identifiable tools for a large number of people already familiar with other video and image processing applications. the second is, for me, fairly critical. what's the point of having video processing code if it can't run at video speed?
Cons:
- confuses the user by adding yet another object to the list of
objects that have to be learned.
- not as flexible for experimental uses
- code duplication (yet another convolution kernel)
- building a consistent set of high level object needs organization
i will actually argue that it will not be confusing at all. certain people can use the higher level object immediately because they recognize it's function as part of a common tool-set.
i agree with the second and third cons
the last point is something that i feel has to be done to GEM anyway. better docs and a well laid out tutorial will go along way toward this.
- Building higher level abstractions:
Pro:
- usability, name corresponds to function (same as with dedicated
external)
- allows for a more consistent hierarchy when patching (e.g. general low level objects are externals, abstractions are more specified objects programmed in pd itself).
- code base is kept small --> easier to debug/maintain/optimize
those are all good points.
Cons:
- loss in performance (how much ?)
- building a consistent set of high level object needs organization
I think the key question is really the loss in performance. This said, I think we should make a folder within GEM for GEM abstractions, and based on the performance issue decide if a new object will be an abstraction or a dedicated external.
as far as performance goes, the current pix_convolve is extremely slow, and needs lots of improvement. there is a great deal of optimization to be done to different kernels: certain types of edge-detection need only luma values thus saving a considerable amount of computation vs a general convolution function. another example is directional motion-blur which only works on a specified axis.
also, the different color-spaces don't handle convolution in exactly the same way. the optimizations can be even greater when the proper color-space is applied to the desired kernel.
For the special case of "blur", I would have been happier if the pix_convolve object gets optimized, and pix_blur built as an abstraction.
my choice for the name [pix_blur] was not the best, since it actually does a motion blur based on feedback rather than a convolution. perhaps it should be renamed to [pix_motionblur]?
The tv_ objects problem: It would be great if we could come up with a way in GEM to store images and do recursion, in order to implement these effects directly as a patch. Can this be done without loosing too much performance ? I think yes.
you are talking about a buffering system? something like [pix_buffer] with [pix_read/write] or tapin/tapout is on my TODO list. i was thinking of making it very simple, and probably even a single object. the control over where images are read in and out would be entirely user controlled as far as reading and writing to the buffer.
the performance question really concerns the overhead of moving memory around. it's much better to do some sort of operation on the data when it has to be moved (the more ops the better), otherwise the whole processing chain stalls while the data is copied from one location to another. in some cases there is no way to avoid this.
cgc
Greetings,
Guenter