FYI: I just subscribed to this list and I am resending this email in case it didn't make it to the list the first time.
On Dec 2, 2004, at 4:40 AM, Johannes M Zmoelnig wrote:
Hans-Christoph Steiner wrote:
coming from Gem too. Maybe the way to do it would be to have a [gemhid] object that outputs the same format as [hid].
(i do not know how to write "have you read all of the postings in this thread?" without appearing to be offensive. i don't know whether you are on gem-dev; so i ask the question without offense)
No offence taken. I am not on gem-dev so I haven't read this thread, and I haven't really done much with Gem, so I am at a double disadvantage.
i really think that the information provided by gem is per-window. as i understand hid it is rather "global", in a sense that you do not know where the information comes from. (well, it comes from "the mouse" of course) now the problem here is that we want multiple windows with distinct grabbing of hid-events (if we want "global" events, i think one should use [hid]). [gemmouse] (and friends) does not support this, as there is no way to specify the window it should listen too. so most naturally (for me) it seems to take pdp's (and others') approach to add an outlet to the [gemwindow] object that gives you this kind of "feedback" from this window. the output of this outlet should be standardized, and most probably the "HID format" would be the best choice (as ben suggested), as it seems to be easily extendible to new features of new hi-devices (e.g. scroll-wheels,...)
Sorry if this seems annoying, but I need to understand the whole problem before giving good advice. The idea behind the [hid] stuff is that you are getting sensor data from the devices, with as little filtering from the OS as possible. Correct me if I am wrong, but the basic goal with Gem seems to be to get pointer position data and clicks within that gemwin, so that's a different thing. That's getting information from the OS about the pointer interacting with the window.
For example, something like keyboard data is not at all related to the pointer, so it doesn't matter whether you getting the events globally (i.e. from [hid]) or locally (i.e. from [gemkeyboard]). The only difference is whether the gemwin has focus or not. But if you can get the focus state from the gemwin, then you could use [hid] to get keyboard events for that specific window. I think this would be more flexible. The same goes for any element of a HID that is not directly represented in the gemwin (axes other than X,Y, mouse wheels, throttles, tablet pen tilt, etc. etc.).
So it seems that Gem should only handle the window interaction data (pointer coords, button clicks in that window), and output when gemwin has focus (maybe it does this already?). The rest can be handled with standard Pd objects (i.e. [hid], or whatever). This idea is a bit raw since I am mired in writing my thesis paper, but I think this direction would ultimately be the right approach. Less duplicated code, fewer different objects to learn, more flexible, etc.
What I haven't understood about the Gem HID objects (gemmouse, gemtablet, etc) is why they need to be distinct objects from the standard Pd HID objects.
sorry i do not understand this. if you complain about Gem's HID objects being unnecessary, you should have written your [hid] some years earlier (before april 1998; i am not sure whether HID was such a buzzword back then)...
I had zero intention of complaining, I was just under the impression that y'all wanted to keep the gem HID objects distinct, and I wanted to understand why. That's it. I have only used Gem a little, and I haven't looked at the code at all, so in order for me to be helpful here, I need to understand what all is going on.
.hc
why we have several objects [gemmouse], [gemtablet], [gemkeyboard] i cannot really tell you. but it seemed a bad idea to output keyboard-information to [gemmouse]...
The only reason I can see is to have absolute coordinates that match up the the gemwin.
these coordinates are relative to the position of the gem-window (not absolute with respect to the desktop-origin).
________________________________________________________________________ ____
There is no way to peace, peace is the way. -A.J. Muste
hi
Hans-Christoph Steiner wrote:
No offence taken.
fine!
Sorry if this seems annoying, but I need to understand the whole problem before giving good advice.
ok. a short summary: there are a couple of objects that get user-input "in the context of the gem-window", namely mouse, keyboard and tablet (i am not sure at all, whether [gemtablet] is limited to the gem-window somehow as i have never used a tablet. probably it is not. as Gem currently supports only one window, all the objects (at least mouse+keyboard) are related to this single window.
now we plan to support multiple windows in the future. user-interaction should be (in my opinion) per-window, especially if (as it is) the retrieved data (like mouse-position) is in relation to that window.
this shouldn't be a real problem with keyboard (and tablet, if (which is what i believe) it is not related to the "window" at all)
The idea behind the [hid] stuff is that you are getting sensor data from the devices, with as little filtering from the OS as possible. Correct me if I am wrong, but the basic goal with Gem seems to be to get pointer position data and clicks within that gemwin, so that's a different thing. That's getting information from the OS about the pointer interacting with the window.
basically true. i have added the [gemkeyname] a while ago, back then when there was only pd's [keyname] which only responded to key-strokes within pd's focus. this is "unfortunate", as we now have to provide a layer of compatibility which (in my opinion) must not depend on anything but Gem and pd (this is: no other externals must be involved)
For example, something like keyboard data is not at all related to the pointer, so it doesn't matter whether you getting the events globally (i.e. from [hid]) or locally (i.e. from [gemkeyboard]). The only
that is correct.
difference is whether the gemwin has focus or not. But if you can get the focus state from the gemwin, then you could use [hid] to get
i am planning to output a "focus" message just because of this.
So it seems that Gem should only handle the window interaction data (pointer coords, button clicks in that window), and output when gemwin has focus (maybe it does this already?). The rest can be handled with standard Pd objects (i.e. [hid], or whatever). This idea is a bit raw since I am mired in writing my thesis paper, but I think this direction would ultimately be the right approach. Less duplicated code, fewer different objects to learn, more flexible, etc.
big words: "standard Pd objects (i.e. [hid]" ;-) anyhow, i agree that not each and every external that makes it's own windows should (have to) handle all the events that appear in that window (especially if they are not really window-related like joystick)
I had zero intention of complaining, I was just under the impression that y'all wanted to keep the gem HID objects distinct, and I wanted to
the only reason is backwards compatibility.. i have no personal liking of [gemkeyboard] at all (although might have been very useful to other people)
conclusions (not necessarily drawn from the babble above) in no order, and partly just my personal opinion:
+MUST: there must be a compatibility layer to the old [gemmouse] and friends (most likely built as abstractions)
+NEED: there *should* be no dependencies on other external libraries (we are going to have a hard time if people have to recompile their kernels because they don't have evdev-support which they need to get a [hid]-based [gemkey] to work again); otoh [hid] would be the choice as it is cross-platform (does it work on irix ?)
+NEED: window-related events (clicks in window, mouse-position, ...) should be handled on a per window basis, presumably as a message-output by the object that represents the window (e.g. [gemwindow]).
+[gemmouse] is window related and would thus be provided by Gem itself (and it is surely the mostly used of all gemhids)
+[gemkeyboard] (or whatever it's exact name is;-)) might still be "subject to change" (as the help-file says) as the output-format is not completely cross-platform, or was: gerard (i think) has submitted a patch that makes linux and windoze keynames be the same; no idea about osX though (does it work at all?); this gives us a lot freedom !
+[gemtablet] might entirely vanish and be substituted by a wrapper around hid. this seems contradictory, but the tablet-code does not work to well, is not cross-platform at all (it worked under NT but i have no idea about modern windozes with modern devices, i have no idea about linux (maybe works, but rather not), and i have no idea about osX (probably doesn't work)), as an object with umpteen ooutlets it is clumsy (sorry Mark). a compatibility abstraction should not be to hard to do.
and just an incomplete list of (proposed) messages that might/should be emitted by a [gemwindow] "mouse" (with x, y, buttons, wheels and stuff); "focus" "offset" (window position) "dimen" (window dimension) probably all information you get at the console when sending an "info"-message should be output as well. i am not sure whether "offset" and "dimen" should be sent automatically whenever they are changed by the user (by dragging the window and resizing it) or whether this should only be retrieved when the object gets polled.
ah long mail hope it makes sense mfg.muzt.ys IOhannes
This whole thing is definitely colored by my view of treating input devices like sensors. But I think this would lead to the most flexible system with the least duplicated code and effort:
- [gemmouse] is not representing the mouse, but only the mouse cursor info. A mouse outputs relative info, not absolute coords. The current [gemmouse] outputs absolute coords of the cursor, whether that cursor is controlled by mouse, tablet, joystick, keyboard, eye tracking, whatever.
- Gem should handle window focus and window position. Everything else can be handled in the regular Pd space, including mouse cursor coords and clicks within the gem window.
So I think that Gem should only output focus and the root location of each window. Then everything else, i.e. getting data from HIDs, would be handled in the Pd space. Then we need to create a [cursor] (or a better name? [ospointer]? [systempointer]? [mousepointer]?) object which gets the mouse cursor coords from the OS. I think that you could easily implement all of the existing [gemmouse] etc. functionality with such a design, and make it work with multiple windows.
.hc
On Dec 3, 2004, at 3:57 AM, IOhannes m zmoelnig wrote:
hi
Hans-Christoph Steiner wrote:
No offence taken.
fine!
Sorry if this seems annoying, but I need to understand the whole problem before giving good advice.
ok. a short summary: there are a couple of objects that get user-input "in the context of the gem-window", namely mouse, keyboard and tablet (i am not sure at all, whether [gemtablet] is limited to the gem-window somehow as i have never used a tablet. probably it is not. as Gem currently supports only one window, all the objects (at least mouse+keyboard) are related to this single window.
now we plan to support multiple windows in the future. user-interaction should be (in my opinion) per-window, especially if (as it is) the retrieved data (like mouse-position) is in relation to that window.
this shouldn't be a real problem with keyboard (and tablet, if (which is what i believe) it is not related to the "window" at all)
The idea behind the [hid] stuff is that you are getting sensor data from the devices, with as little filtering from the OS as possible. Correct me if I am wrong, but the basic goal with Gem seems to be to get pointer position data and clicks within that gemwin, so that's a different thing. That's getting information from the OS about the pointer interacting with the window.
basically true. i have added the [gemkeyname] a while ago, back then when there was only pd's [keyname] which only responded to key-strokes within pd's focus. this is "unfortunate", as we now have to provide a layer of compatibility which (in my opinion) must not depend on anything but Gem and pd (this is: no other externals must be involved)
For example, something like keyboard data is not at all related to the pointer, so it doesn't matter whether you getting the events globally (i.e. from [hid]) or locally (i.e. from [gemkeyboard]). The only
that is correct.
difference is whether the gemwin has focus or not. But if you can get the focus state from the gemwin, then you could use [hid] to get
i am planning to output a "focus" message just because of this.
So it seems that Gem should only handle the window interaction data (pointer coords, button clicks in that window), and output when gemwin has focus (maybe it does this already?). The rest can be handled with standard Pd objects (i.e. [hid], or whatever). This idea is a bit raw since I am mired in writing my thesis paper, but I think this direction would ultimately be the right approach. Less duplicated code, fewer different objects to learn, more flexible, etc.
big words: "standard Pd objects (i.e. [hid]" ;-) anyhow, i agree that not each and every external that makes it's own windows should (have to) handle all the events that appear in that window (especially if they are not really window-related like joystick)
I had zero intention of complaining, I was just under the impression that y'all wanted to keep the gem HID objects distinct, and I wanted to
the only reason is backwards compatibility.. i have no personal liking of [gemkeyboard] at all (although might have been very useful to other people)
conclusions (not necessarily drawn from the babble above) in no order, and partly just my personal opinion:
+MUST: there must be a compatibility layer to the old [gemmouse] and friends (most likely built as abstractions)
+NEED: there *should* be no dependencies on other external libraries (we are going to have a hard time if people have to recompile their kernels because they don't have evdev-support which they need to get a [hid]-based [gemkey] to work again); otoh [hid] would be the choice as it is cross-platform (does it work on irix ?)
+NEED: window-related events (clicks in window, mouse-position, ...) should be handled on a per window basis, presumably as a message-output by the object that represents the window (e.g. [gemwindow]).
+[gemmouse] is window related and would thus be provided by Gem itself (and it is surely the mostly used of all gemhids)
+[gemkeyboard] (or whatever it's exact name is;-)) might still be "subject to change" (as the help-file says) as the output-format is not completely cross-platform, or was: gerard (i think) has submitted a patch that makes linux and windoze keynames be the same; no idea about osX though (does it work at all?); this gives us a lot freedom !
+[gemtablet] might entirely vanish and be substituted by a wrapper around hid. this seems contradictory, but the tablet-code does not work to well, is not cross-platform at all (it worked under NT but i have no idea about modern windozes with modern devices, i have no idea about linux (maybe works, but rather not), and i have no idea about osX (probably doesn't work)), as an object with umpteen ooutlets it is clumsy (sorry Mark). a compatibility abstraction should not be to hard to do.
and just an incomplete list of (proposed) messages that might/should be emitted by a [gemwindow] "mouse" (with x, y, buttons, wheels and stuff); "focus" "offset" (window position) "dimen" (window dimension) probably all information you get at the console when sending an "info"-message should be output as well. i am not sure whether "offset" and "dimen" should be sent automatically whenever they are changed by the user (by dragging the window and resizing it) or whether this should only be retrieved when the object gets polled.
ah long mail hope it makes sense mfg.muzt.ys IOhannes
GEM-dev mailing list GEM-dev@iem.at http://iem.at/cgi-bin/mailman/listinfo/gem-dev
________________________________________________________________________ ____
There is no way to peace, peace is the way. -A.J. Muste
...switching back to multiple_window GEM, I figured out a simple way to get the window bounds on OSX! So, I'm flying thru the rest of the compile and stop at pix_2grey, which has a simple typo that led me to discover a bunch of MMX code? Not that I need or can test it personally, but when are you going commit this stuff to the HEAD? I would've figured that MMX code would go into HEAD before multiple_window?
...Also running into it in pix_gain: I'll commit my fixes when I get home...but it'd be nice to have everything up to date :-/
jamie
james tittle wrote:
...switching back to multiple_window GEM, I figured out a simple way to get the window bounds on OSX! So, I'm flying thru the rest of the compile and stop at pix_2grey, which has a simple typo that led me to discover a bunch of MMX code? Not that I need or can test it personally, but when are you going commit this stuff to the HEAD? I would've figured that MMX code would go into HEAD before multiple_window?
yes, we should merge the SIMD-branch with MAIN as soon as possible i just wanted some of you AltiVec-guys check it out and do a little testing (as i have renamed functions a bit to make naming more consistent) as soon as you give your ok i will merge them and probably merge multiple_window with SIMD too (just to ensure we are working on the latest code-base)
mfg.a.sdr IOhannes
Johannes M Zmoelnig wrote:
james tittle wrote:
yes, we should merge the SIMD-branch with MAIN as soon as possible
oh and we should merge the vertex-arrays too and release.
btw, 2 questions for the vertex-array stuff (now that you have started to work on multiple_window ;-)) do the vertex_tables work all-right ?
and i have changed the width of all of the arrays to 4 (this is: x/y/z/w but also u/v/0/0) to make them more uniformly. probably someone with a good profiler (chris ?) could check whether the performance loss is really that big or if we can accept that in order to be more flexible (esp. in terms of coding as it is so much easier to just have consider one width)
mfg.a.dr IOhannes
On Dec 14, 2004, at 3:29 AM, Johannes M Zmoelnig wrote:
and i have changed the width of all of the arrays to 4 (this is: x/y/z/w but also u/v/0/0) to make them more uniformly. probably someone with a good profiler (chris ?) could check whether the performance loss is really that big or if we can accept that in order to be more flexible (esp. in terms of coding as it is so much easier to just have consider one width)
The data is not general enough to be treated this way and it's unlikely to save any coding effort for more than the most trivial objects. They don't even have the same ranges for valid data. For example, vertex data is not restricted to any coordinate values, but color info is clamped at 1.0, so some sort of scaling would have to take place for these to be exchanged (I would say the color info should be unsigned chars anyway). Also, an array width of 3 for normals is not valid according to the RedBook, so that is going to require truncation. Finally, these objects should have the highest level of optimizations possible as dealing with even a 10,000 poly object at a decent framerate is going to murder most any CPU.
So my vote is that the array widths should go back to there natural sizes as the very minor and rare benefits of equal lengths outweigh the negatives.
cgc
chris clepper wrote:
chars anyway). Also, an array width of 3 for normals is not valid according to the RedBook, so that is going to require truncation.
so what is then the valid array width for normals ?
So my vote is that the array widths should go back to there natural sizes as the very minor and rare benefits of equal lengths outweigh the negatives.
i understand this, however i would like to see some numbers on how much the increase of data really affects the performance. i mean, it is clear that there is a lot of space for SIMD-optimization; but what interests me is, how much it costs more to transfer 4 instead of 3 floats from ram to the gfx-card.
i really don't stick to this interchangibility thing (although i like the idea), but i would like to hear you say "on my machine using 4 floats as texture-coordinates eats half of the cpu" or something like that before we fix this forever.
and i am not sure at all, whether we gain something (but memory, which in turns decreases cpu-load) by using uchar instead of float for colors. wouldn't most gfx-cards convert them back to floats ?
while pixes are the bridge to pdp, i guess vertex-arrays might be the bridge to GridFlow.
mfg.a.sdr IOhannes
On Dec 14, 2004, at 10:27 AM, Johannes M Zmoelnig wrote:
chris clepper wrote:
chars anyway). Also, an array width of 3 for normals is not valid according to the RedBook, so that is going to require truncation.
so what is then the valid array width for normals ?
...I think here he meant 3 is valid, but 4 is not...just the same as Index and colors have a certain size
and i am not sure at all, whether we gain something (but memory, which in turns decreases cpu-load) by using uchar instead of float for colors. wouldn't most gfx-cards convert them back to floats ?
...sure, everything on the gpu is float (at least color/vertex-wise)...
while pixes are the bridge to pdp, i guess vertex-arrays might be the bridge to GridFlow.
...wha? GridFlow doesn't know anything about 3d atm...sure, I can see use of gridflow's matrix manipulations: it could be really good for something like the shapeSynth (and that was my original plan, to implement it in pd/GEM and then again in pd/GEM/Gridflow, just as a comparison/exercise)
...just to throw something else into the ring, I've been imagining a series of "[vp_*]"'s that would use vertex programs to manipulate data that is already on the card (in the somewhat the same manner as the [vertex_sub/add/whatever] stuff): then we don't have a choice of what format! But, we'd be doing all that computation on the card, and the cpu can do something else :-)
jamie
james tittle wrote:
On Dec 14, 2004, at 10:27 AM, Johannes M Zmoelnig wrote:
...I think here he meant 3 is valid, but 4 is not...just the same as Index and colors have a certain size
but what for do we have the "stride"-argument then ??
while pixes are the bridge to pdp, i guess vertex-arrays might be the bridge to GridFlow.
...wha? GridFlow doesn't know anything about 3d atm...sure, I can see use of gridflow's matrix manipulations: it could be really good for
yes, that is what i meant: GridFlow is about matrix-manipulation of arrays (sorry matiu if this is wrong...) and vertex-arrays are too. but it was just a wild idea.
...just to throw something else into the ring, I've been imagining a series of "[vp_*]"'s that would use vertex programs to manipulate data that is already on the card (in the somewhat the same manner as the [vertex_sub/add/whatever] stuff): then we don't have a choice of what format!
ah, so much things to do...
mfg.asd.r IOhannes
On Tue, 14 Dec 2004, james tittle wrote:
On Dec 14, 2004, at 10:27 AM, Johannes M Zmoelnig wrote: ...wha? GridFlow doesn't know anything about 3d atm...sure, I can see use of gridflow's matrix manipulations:
In GridFlow, the convention for representing a M-sided polygon embedded in a N-dimensional space, is to use a grid of size (M,N) where each row is a vertex. Edges go from each vertex to the next one, modulo M.
I haven't made any operations specific to polygons, except [@draw_polygon], which is crummy. Many objects of GridFlow do interesting things on polygons. One starting point would be to try out things with [@remap_image] and make the analogy between that context (the indexmap context) and the polygon context, because both are geometric.
E.g. [@ + 42] on a pixmap does brightness adjustment, but [@ + 42] on an indexmap does a translation (it moves the coordinates). However [@ + 42] on polygons also does a translation, because although indexmaps and polygons are different, they are also much the same.
_____________________________________________________________________ Mathieu Bouchard -=- Montréal QC Canada -=- http://artengine.ca/matju
On Tue, 14 Dec 2004, Johannes M Zmoelnig wrote:
while pixes are the bridge to pdp, i guess vertex-arrays might be the bridge to GridFlow.
There ought to be two bridges to gridflow, one for pixes and one for vertex-arrays. Well at least I don't know why there shouldn't be one for pixes... At least greyscale and rgb would be supported.
For the yuv422 case, it could be mapped to a grid of size (height,width,2), where the two channels are Y and UV. The UV plane has U on even-numbered columns and V on odd-numbered columns. That's afaict the yuv422 model i will use if i ever need to.
Another possible mapping for yuv422 could be to duplicate the UV channels to make it yuv444; then that maps directly to something that can be used most naturally in GridFlow (i have already used yuv-based effects in practice in GF).
The yuv444 i am using is unsigned, but frankly, most of the time I translate to u8:i8:i8 (y unsigned, uv signed) because then the transform is homogeneous linear and so the greys are at the origin of the uv plane. So in the end i'd support both.
But I think RGB & Greyscale will be an excellent start...
_____________________________________________________________________ Mathieu Bouchard -=- Montréal QC Canada -=- http://artengine.ca/matju
On Dec 14, 2004, at 10:05 AM, chris clepper wrote:
On Dec 14, 2004, at 3:29 AM, Johannes M Zmoelnig wrote:
and i have changed the width of all of the arrays to 4 (this is: x/y/z/w but also u/v/0/0) to make them more uniformly.
So my vote is that the array widths should go back to there natural sizes as the very minor and rare benefits of equal lengths outweigh the negatives.
...chris and I talked about this last week at length, and I also believe that we should not inflate the data to potentially allow flexibility... ...first off, the original idea of vertex_array was to add a way of manipulating vertex arrays on the cpu, and optimize the uploading of the data (as we have with vertex_array_object/vertex_buffer_object)...
...second, the possible data's are not mutually interchangeable: like chris said, colors will always be upload as 0-1.0, whereas normals and vertices can be anything +/-...
...third, there are other arrays that haven't been implemented (SecondaryColor, Index, FogCoord, EdgeFlag): I admit I don't know what SecondaryColor is really good for, but if we were to inflate each of these to 4 floats then we're really talking about bloat...
...in essence, I think that it'd be better to have an optimized pathway to deal with specific data, and let the user massage the data into the different formats on their own (or provide abstractions or objects that do this)...
...as far as numbers go, I can try to do some benchmarking next weekend...
jamie
james tittle wrote:
...chris and I talked about this last week at length, and I also believe that we should not inflate the data to potentially allow flexibility...
...third, there are other arrays that haven't been implemented (SecondaryColor, Index, FogCoord, EdgeFlag): I admit I don't know what SecondaryColor is really good for, but if we were to inflate each of these to 4 floats then we're really talking about bloat...
well, but probably the generic solution would be better for this, as we could just use _any_ array and upload it as EdgeFlag (whatever that is) so we would only need (say) 4 arrays to cover everything.
...in essence, I think that it'd be better to have an optimized pathway to deal with specific data, and let the user massage the data into the different formats on their own (or provide abstractions or objects that do this)...
might be good too; probably it would be ok to have several arrays of different types (e.g. width-3, or uchar) and than let the user choose which one is which.
...as far as numbers go, I can try to do some benchmarking next weekend...
great
mfg.as.dr IOhannes
On Dec 14, 2004, at 4:29 AM, Johannes M Zmoelnig wrote:
Johannes M Zmoelnig wrote:
james tittle wrote: yes, we should merge the SIMD-branch with MAIN as soon as possible
oh and we should merge the vertex-arrays too and release.
...well, as far as I can tell the SIMD stuff should work (tho I expect a few more typos): I can grab this probably tonight...otoh, we've gotta straighten out some things with vertex-arrays, but I'll talk about that in another email...
btw, 2 questions for the vertex-array stuff (now that you have started to work on multiple_window ;-)) do the vertex_tables work all-right ?
...haven't actually gotten around to testing it: but it compiles fine! Thanks for getting this in :-)
jamie
-
chris clepper -
Hans-Christoph Steiner -
IOhannes m zmoelnig -
james tittle -
Johannes M Zmoelnig
-
Mathieu Bouchard