I'm pretty determined to work through this! :-)

I thought I'd write a workaround to keep this conversation moving: My plan was to write the video to a buffer with [pix_buffer], then read one frame later and apply [pix_rtx]. I've done this workaround before for similar situations.

But I got stuck before I even got to [pix_rtx]. Please see the attached patch.

In the patch you'll see 3 toggles, all of which I would have expected to show a rotated image textured onto a rectangle. Only one of them does. They are numbered right to left.

Box 1 shows the output of [pix_buffer_write]. That shows a rotated image. Fine so far.

Box 2 shows the output of [rotateXYZ] before [pix_buffer_write]. That shows a rotated white box with no image textured. Why?

Box 3 shows the output of [pix_buffer_read]. That shows an image but no rotation. Why? This one is especially troublesome to me.

I'm trying to make my questions as specific as possible. Please understand I have looked at plenty of examples of [rotateXYZ] and I understand the object and have made full use of it. And I have previously read about [pix_separator] and know the archives that refer to it. There's something else going on here about dataflow with Gem that I don't get. I can make up 10 other examples with 10 other objects if I have to.

-John

John Harrison wrote:

Thanks Hans. I know the example patch already and I have [pix_rtx] working fine inasfar as I can get it to do what the example patch already shows. I think my question is more about data flow and Gem. My patch and [pix_rtx} are just an example of a more basic misunderstanding I believe I have which has not been cleared up despite many hours of work in Gem.

Could somebody explain to me my [pix_rtx] in my example patch appears to be processed before the rotation instead of after?

-John

On Tue, Mar 24, 2009 at 9:33 PM, Hans-Christoph Steiner <hans@eds.org mailto:hans@eds.org> wrote:

I think this is an appropriate place to discuss gem.  In the Help
browser, check out:

examples -> Gem -> 04.video -> 04.videoRTX.pd

I had to do some trickery to get it working OK.  Check the
archives for details, I remember some discussion about all the
separators.

.hc


On Mar 24, 2009, at 3:35 PM, John Harrison wrote:

    Attached is a small example patch of how I just don't get
    Gem...still...

    [pix_rtx] has a steady wave that normally flows from left to
    right.

    I would have thought that the attached patch shows me rotating
    the image first, then applying [pix_rtx]. So it would stand to
    reason in my mind I would see a rotated image with [pix_rtx]
    flowing from left to right.

    But that isn't the result I see. The patch has [pix_rtx]
    flowing from right to left now. It is as if [pix_rtx] is
    applied *before* the rotation instead of *after* as I would
    have expected.

    Grasping at straws, I have tried [pix_separator] between just
    about every object, but that makes no difference.

    What am I misunderstanding that makes the behavior of the
    patch make sense? And...how would I get [pix_rtx] to flow from
    left to right on a mirrored image?

    -John

    P.S. Are questions like this better on the Gem-dev list?
    That's a developer list but at the same time I feel a bit
    awkward putting too many Gem questions on a Pd list

    #N canvas 962 171 305 227 10;
    #X obj 29 -58 gemhead;
    #X obj 29 -35 pix_video;
    #X obj 29 44 pix_texture;
    #X obj 28 72 rectangle 4 3;
    #X obj 159 9 gemwin;
    #X msg 136 -34 create \, 1;
    #X msg 214 -33 destroy;
    #X obj 135 -57 loadbang;
    #X obj 29 17 pix_rtx;
    #X obj 29 -10 rotateXYZ 0 180 0;
    #X connect 0 0 1 0;
    #X connect 1 0 9 0;
    #X connect 2 0 3 0;
    #X connect 5 0 4 0;
    #X connect 6 0 4 0;
    #X connect 7 0 5 0;
    #X connect 8 0 2 0;
    #X connect 9 0 8 0;
    _______________________________________________
    Pd-list@iem.at <mailto:Pd-list@iem.at> mailing list
    UNSUBSCRIBE and account-management ->
    http://lists.puredata.info/listinfo/pd-list




----------------------------------------------------------------------------

There is no way to peace, peace is the way.       -A.J. Muste

On Wed, Mar 25, 2009 at 10:22 AM, marius schebella < marius.schebella@gmail.com> wrote:

hi john, I just looked at your patches, here's what I think:

1. pix_buffer_write seems to copy and clear the pix buffer. so if your

pix_video is connected to pix_buffer_write it is gone and you don't see it anymore (rectangle stays white). I am not sure if this is a bug, it seems strange to me, too. but as soon as you disconnect the pix_buffer_write the image will be shown on the rotated rectangle.

Yes I also noticed that disconnecting the [pix_buffer_write] fixes the problem. i see that sort of thing in Gem all the time ([pix_mix] for example.) But wouldn't a [pix_separator] fix this then? It doesn't...

2. the rotateXYZ does not rotate the pixels, but only the geometry

that this texture is mapped to (the rectangle). so in your left example you go from gemhead through buffer_read, texture to the rectangle. in that line no rotationXYZ is applied to the rectangle. marius.

So why does the right example work? Because the rotation happens after the [pix_buffer_write] even though [pix_buffer_write] is later in the rendering chain? Or perhaps the rotation is ignored by [pix_buffer_write]? And at the same time [pix_buffer_write] sends the output of the rotation as a "passthrough" on its outlet? And also you are saying it would be normal and expected for the output of [pix_buffer_write] to be different than what [pix_buffer_write] is actually storing? If that isn't the case, example 1 and example 3 should produce the same results.

Perhaps I need some rules about how Gem is different than Pd in terms of dataflow. So far it seems to me that a rendering chain in Gem shows conections but does not reveal the order in which these connections are processed i.e. if 3 objects are all connected to each other in a rendering chain in Gem i.e.

[Gemhead] | [object 1] | [object 2] | [object 3]

we cannot conclude that the output of [object 1] feeds the input of [object 2] and the output of [object 2] feeds the input of [object 3]. We only know that the data will be processed by objects 1, 2 and 3 but not in what order they will be processed.

Further perhaps we cannot conclude that the output of an object is actually the result of the object having processed the data. Demonstrating this: in my patch example, [pix_buffer_write] outputs different data than it stores, or examples 1 and 3 would be the same.

And this leads me to a more general statement of my original question: how can I determine the order in which objects are processed in Gem?

And my specific example which started this thread: how would one apply a rotation to an image, then apply [pix_rtx] to the rotated image?

-John

2009/3/24 John Harrison john.harrison@alum.mit.edu:

...

Attached is a small example patch of how I just don't get

Gem...still...

...

[pix_rtx] has a steady wave that normally flows from left to right.

I would have thought that the attached patch shows me rotating the image first, then applying [pix_rtx]. So it would stand to reason in my mind I would see a rotated image with [pix_rtx] flowing from left to right.

But that isn't the result I see. The patch has [pix_rtx] flowing from

right

...

to left now. It is as if [pix_rtx] is applied *before* the rotation

instead

...

of *after* as I would have expected.

Grasping at straws, I have tried [pix_separator] between just about every object, but that makes no difference.

What am I misunderstanding that makes the behavior of the patch make

sense?

...

And...how would I get [pix_rtx] to flow from left to right on a mirrored image?

-John

P.S. Are questions like this better on the Gem-dev list? That's a

developer

...

list but at the same time I feel a bit awkward putting too many Gem questions on a Pd list

#N canvas 962 171 305 227 10; #X obj 29 -58 gemhead; #X obj 29 -35 pix_video; #X obj 29 44 pix_texture; #X obj 28 72 rectangle 4 3; #X obj 159 9 gemwin; #X msg 136 -34 create , 1; #X msg 214 -33 destroy; #X obj 135 -57 loadbang; #X obj 29 17 pix_rtx; #X obj 29 -10 rotateXYZ 0 180 0; #X connect 0 0 1 0; #X connect 1 0 9 0; #X connect 2 0 3 0; #X connect 5 0 4 0; #X connect 6 0 4 0; #X connect 7 0 5 0; #X connect 8 0 2 0; #X connect 9 0 8 0;

---

Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list

This is extremely helpful. I'm starting to "get" it. Comments/questions inline.

chris clepper wrote:

There are two types of objects in GEM: pix and OpenGL.

Pix objects do work in the top to bottom manner like Pd DSP objects.

would [pix_coordinate] be an exception to this? I've been playing with it and it seems to behave the way you have described OpenGL objects and not GEM objects. I read the help patch about [pix_texture] reassigning the coordinate values, but that still didn't explain all the behavior I was seeing.

The convention in GEM is to put the GL objects after the pix_ ones showing that once the pix_ processes are done on the CPU it is time for the GL processes on the GPU to start.

I understand you to be saying that the GL processes will always be applied after the pix_ processes. If that is the case, then it sounds like there is no way to have [rotateXYZ] applied before [pix_rtx]. Makes sense.

For the more general case, is it correct that there is no way in GEM to give an arbitrary rotation of an image as input to a pix_ object since there is no Gem pix_ object with arbitrary rotation function?

Figuring that might be the case, I tried to build [pix_rotate] using [pix_coordinate]...and this led me to the question about about [pix_coordinate]

Using pdp_rotate, pix_2gem and gem2pdp, I did successfully build a pix_ rotator. On the chance it might be helpful to see, I attached a demo patch which feeds a rotated video stream to [pix_rtx] then rotates the stream back to the way it was. While it more-or-less works it seems a bit scabby. Is there a better way?

There are lots of exceptions to Pd rules in GEM and there is really no way around them. It is kind of like learning English - I before E except after C, excepting all of those words that ignore the rule.

As long as I can make sense of what the rules are, which objects break them, and some rough idea as to why, I'm cool with that.

Thank you again for your help. As Hans suggests, I'd like to find a way to help organize, then share this information, whether it be on the wiki or in some other meaningful way.

-John

#N canvas 65 149 758 508 10; #X obj 95 44 gemhead; #X obj 95 67 pix_video; #X obj 235 386 pix_texture; #X obj 234 405 rectangle 4 3; #X obj 48 12 gemwin; #X msg 25 -31 create , 1; #X msg 103 -30 destroy; #X obj 24 -54 loadbang; #X obj 229 245 pix_rtx; #X obj 166 150 pdp_rotate; #X obj 184 120 pix_2pdp; #X floatatom 244 96 5 0 0 0 - - -; #X obj 167 172 pdp2gem; #X obj 116 153 gemhead; #X obj 176 96 t a b; #X msg 151 24 colorspace YUV; #X obj 149 3 loadbang; #X obj 235 331 pdp_rotate; #X obj 253 301 pix_2pdp; #X obj 236 353 pdp2gem; #X obj 185 334 gemhead; #X obj 245 277 t a b; #X obj 304 202 * -1; #X msg 231 217 set; #X obj 246 122 t f b; #X text 284 96 <- change angle of "wave" of rtx; #X connect 0 0 1 0; #X connect 1 0 14 0; #X connect 2 0 3 0; #X connect 5 0 4 0; #X connect 6 0 4 0; #X connect 7 0 5 0; #X connect 8 0 21 0; #X connect 9 0 12 0; #X connect 10 1 9 0; #X connect 11 0 24 0; #X connect 12 0 8 0; #X connect 13 0 12 0; #X connect 14 0 10 0; #X connect 14 1 10 0; #X connect 15 0 1 0; #X connect 16 0 15 0; #X connect 17 0 19 0; #X connect 18 1 17 0; #X connect 19 0 2 0; #X connect 20 0 19 0; #X connect 21 0 18 0; #X connect 21 1 18 0; #X connect 22 0 17 1; #X connect 23 0 8 0; #X connect 24 0 9 1; #X connect 24 0 22 0; #X connect 24 1 23 0;

This is fantastic! Thank you Marius for taking the time to build these patches.

rtx_sch2.pd exactly answered my question and shows beautifully how to perform a set of OpenGL operations on the content of some pix_ data before applying more pix_ operations. The patch makes total sense.

I'm still playing with it to make sure I fully grasp all parts. I really appreciate this help.

-John

marius schebella wrote:

hi, I think your pix_coordinate idea was not that bad (see attached patch). but that is probably not what you want?? on the other hand, using pdp_rotate and converting twice is really eating up a lot of cpu. pdp is a different world again and the bridge between pdp and gem is buggy (your patch crashed my computer for example). but again, as chris said. there is a difference between rotating the content of the pix data and rotating the geometry that this data is mapped to. so the second patch shows, what I think you really want. rotate an image and then feed this into your pix_rtx. marius.

John Harrison wrote:

...

This is extremely helpful. I'm starting to "get" it. Comments/questions inline.

chris clepper wrote:

...

There are two types of objects in GEM: pix and OpenGL. Pix objects do work in the top to bottom manner like Pd DSP objects.

would [pix_coordinate] be an exception to this? I've been playing with it and it seems to behave the way you have described OpenGL objects and not GEM objects. I read the help patch about [pix_texture] reassigning the coordinate values, but that still didn't explain all the behavior I was seeing.

...

The convention in GEM is to put the GL objects after the pix_ ones showing that once the pix_ processes are done on the CPU it is time for the GL processes on the GPU to start.

I understand you to be saying that the GL processes will always be applied after the pix_ processes. If that is the case, then it sounds like there is no way to have [rotateXYZ] applied before [pix_rtx]. Makes sense.

For the more general case, is it correct that there is no way in GEM to give an arbitrary rotation of an image as input to a pix_ object since there is no Gem pix_ object with arbitrary rotation function?

Figuring that might be the case, I tried to build [pix_rotate] using [pix_coordinate]...and this led me to the question about about [pix_coordinate]

Using pdp_rotate, pix_2gem and gem2pdp, I did successfully build a pix_ rotator. On the chance it might be helpful to see, I attached a demo patch which feeds a rotated video stream to [pix_rtx] then rotates the stream back to the way it was. While it more-or-less works it seems a bit scabby. Is there a better way?

...

There are lots of exceptions to Pd rules in GEM and there is really no way around them. It is kind of like learning English - I before E except after C, excepting all of those words that ignore the rule.

As long as I can make sense of what the rules are, which objects break them, and some rough idea as to why, I'm cool with that.

Thank you again for your help. As Hans suggests, I'd like to find a way to help organize, then share this information, whether it be on the wiki or in some other meaningful way.

-John