Hi Miller, hi all, i find that one of the most severe drawbacks concerning the PD interface is the poor information a gui object can get of the interaction status. As far as i could explore, there's is no idle mouseover position information (only click and drag), no mouse up, no mouse wheel and, and less important, only incomplete information on key modifiers (like alt etc.). Although this is all available from the TCL/TK side, directly interfacing TCL/TK (which requires intercepting the PD<->TK communication) could mean possible inportability with future PD versions and problems with other externals also using such hacks.
The desirable solution would be a interaction callback provided by PD to let a GUI object collect all possible interaction information. The latter can be obtained by e.g. using the following TCL/TK statements:
t_canvas *canvas; t_symbol *bindsym; sys_vgui("bind .x%x.c <Motion> {pd %s %s %%x %%y %%s \;}\n",canvas,bindsym,gensym("motion")); sys_vgui("bind .x%x.c <ButtonPress> {pd %s %s 1 %%x %%y %%b %%s \;}\n",canvas,bindsym,gensym("button")); sys_vgui("bind .x%x.c <ButtonRelease> {pd %s %s 0 %%x %%y %%b %%s \;}\n",canvas,bindsym,gensym("button")); sys_vgui("bind .x%x.c <MouseWheel> {pd %s %s %%x %%y %%s %%D \;}\n",canvas,bindsym,gensym("wheel")); sys_vgui("bind .x%x.c <KeyPress> {pd %s %s 1 %%k %%A %%N %%s \;}\n",canvas,bindsym,gensym("key")); sys_vgui("bind .x%x.c <KeyRelease> {pd %s %s 0 %%k %%A %%N %%s \;}\n",canvas,bindsym,gensym("key"));
Note also, that contrary to the PD.TK, where all modifiers are bound separately, here the %s variable is used.
Is it foreseeable that such a callback be included in upcoming versions of PD or should i install a temporary hack?
best greetings, Thomas
btw, Thomas, have you got any reply to this? I would need to find a way of getting info from the gui layer, first in order to clone MouseState and mousefilter, then, possibly, with some other clones too -- although for mouse classes I would probably use a by-request method (polling), using something like "winfo pointerxy".
Yet I do not fully understand your code examples. With replacing tcl/tk bindings of gui events to a canvas (those defined in pd.tk), do you mean a canvas containing your gui object, or a canvas created by that object? If the former, are you going to pd_bind your gui object to the same symbol a containing canvas is bound to (declaring a dummy anything method, I guess)? Or, rather, pd_binding to a different symbol, then resending the gui messages you got in your object, up to a containing canvas?
Krzysztof
Thomas Grill wrote: ...
The desirable solution would be a interaction callback provided by PD to let a GUI object collect all possible interaction information. The latter can be obtained by e.g. using the following TCL/TK statements:
t_canvas *canvas; t_symbol *bindsym; sys_vgui("bind .x%x.c <Motion> {pd %s %s %%x %%y %%s \;}\n",canvas,bindsym,gensym("motion"));
(perhaps bindsym->s_name and "motion"?)
Hi Krzysztof,
btw, Thomas, have you got any reply to this? I would need to find a way of getting info from the gui layer, first in order to clone MouseState and mousefilter, then, possibly, with some other clones too -- although for mouse classes I would probably use a by-request method (polling), using something like "winfo pointerxy".
No, no reply.... so i'm trying to get from pd what i can without having to use bad/non-portable tricks. That is: - key down and up .... this is generally really dirty stuff in PD, there are no scan codes in TCL/TK, just ascii and extended key names - mouse down and mouse dragging (also outside the object rectangle) - idle mouse position
I don't know if there is a way to get the TCL/TK mouse state by polling, there seem to be just mouse events... but that's not too bad - one can cache the info.
Yet I do not fully understand your code examples. With replacing tcl/tk bindings of gui events to a canvas (those defined in pd.tk), do you mean a canvas containing your gui object, or a canvas created by that object? If the former, are you going to pd_bind your gui object to the same symbol a containing canvas is bound to (declaring a dummy anything method, I guess)? Or, rather, pd_binding to a different symbol, then resending the gui messages you got in your object, up to a containing canvas?
These were just examples how PD itself could make use of all the possible information. Looking at PD.TK this is not fully exploited. I can hardly use these code fragments for flext as they would intercept all TCL/TK events. Resending them or modifying PD.TK seems to be non-portable to me as it is very likely to collide with other externals making use of such tricks.
Generally i have to say that i'm (after a short recovery) more and more fed up with TCL/TK. Although it's not a bad system i consider it unsuitable for a multi-media environment. It's a dead-end street. Just had a deeper look at Max/Jitter. It's just incredible.
greetings, Thomas
Just had a deeper look at Max/Jitter. It's just incredible.
How does Jitter compare to GEM (or framestein which I never have used...)
Jitter is a one-in-all system (pixel, OpenGL, Quicktime) with a very general multi-dimensional multi-plane matrix formalism. It's extremely flexible, fast (much faster than NATO for example) and well-documented, and of course, all views can be embedded into the patcher which allows for very elaborated interfaces. It comes with an own SDK for external programming (which is not complete yet), which extends the normal Max object SDK. There is also a new extension to the normal object formalism in the form of "attributes" which can be set by messages (and also directly in the object command line) and also be queried (dumped out). In the next release of flext this feature will also be available for flext externals for PD.
greetings, Thomas
hi Thomas,
Thomas Grill wrote: ...
I don't know if there is a way to get the TCL/TK mouse state by polling, there seem to be just mouse events... but that's not too bad - one can cache the info.
actually I need only mouse _position_ polling (or, strictly speaking, sampling), which is available by calling winfo pointerxy.
Otoh, I do need to be notified about state changes of the mouse _buttons_ using the usual tcl way of binding events -- and what I do not know, is how to plug my object into the original flow of events destined for a canvas (or, rather, all canvases)... Any idea?
...
Generally i have to say that i'm (after a short recovery) more and more fed up with TCL/TK. Although it's not a bad system i consider it unsuitable for a multi-media environment. It's a dead-end street.
:)
Krzysztof
am 16.10.2002 15:21 Uhr schrieb Krzysztof Czaja unter czaja@chopin.edu.pl:
hi Thomas,
Thomas Grill wrote: ...
I don't know if there is a way to get the TCL/TK mouse state by polling, there seem to be just mouse events... but that's not too bad - one can cache the info.
actually I need only mouse _position_ polling (or, strictly speaking, sampling), which is available by calling winfo pointerxy.
Hmmm, i don't understand the advantage of polling compared to position events. If you want to poll from TCL/TK you would have to trigger a TK function and get the values via a bound message (but you know that). So using the mouse events, cache the position and do the polling (if necessary) internally seems more convenient to me.
and what I do not know, is how to plug my object into the original flow of events destined for a canvas (or, rather, all canvases)... Any idea?
I guess in that case you would have to modify the event handlers in PD.TK, which is not very elegant.
best regards, Thomas
Hi Thomas, Krzysztof,
We had several discussions already about redesigning the pd<->GUI comunication, I think that this in general is starting to be a bottleneck.
In my opinion more of the GUI stuff should be done on the GUI side, and not from within pd,... but I might be completely off topic with this ?
Guenter
On Wed, 16 Oct 2002, Thomas Grill wrote:
am 16.10.2002 15:21 Uhr schrieb Krzysztof Czaja unter czaja@chopin.edu.pl:
hi Thomas,
Thomas Grill wrote: ...
I don't know if there is a way to get the TCL/TK mouse state by polling, there seem to be just mouse events... but that's not too bad - one can cache the info.
actually I need only mouse _position_ polling (or, strictly speaking, sampling), which is available by calling winfo pointerxy.
Hmmm, i don't understand the advantage of polling compared to position events. If you want to poll from TCL/TK you would have to trigger a TK function and get the values via a bound message (but you know that). So using the mouse events, cache the position and do the polling (if necessary) internally seems more convenient to me.
and what I do not know, is how to plug my object into the original flow of events destined for a canvas (or, rather, all canvases)... Any idea?
I guess in that case you would have to modify the event handlers in PD.TK, which is not very elegant.
best regards, Thomas
PD-dev mailing list PD-dev@iem.kug.ac.at http://iem.kug.ac.at/cgi-bin/mailman/listinfo/pd-dev
Hi Günther,
We had several discussions already about redesigning the pd<->GUI comunication, I think that this in general is starting to be a bottleneck.
In my opinion more of the GUI stuff should be done on the GUI side, and not from within pd,... but I might be completely off topic with this ?
i think you're absolutely right with that and the flext_gui programming layer is an attempt to move GUI programming away from the PD kernel. However, i'm not undertaking attempts to make a new patcher system. The main problem in my view is the difficulty of embedding non-TK graphical objects into patchers. Attempts to do so (and i've done some experiments, following the way framestein does it) are merely bad hacks and hardly as smooth as desired. As i assume that only a very small fraction of users runs the kernel on another machine as the GUI i don't see a real necessity for the socket based client-server design, but more for a clean and fast portable multithreaded graphical system derived from the native OS graphical interface.
best greetings, Thomas
On Wed, 16 Oct 2002, Thomas Grill wrote:
As i assume that only a very small fraction of users runs the kernel on another machine as the GUI i don't see a real necessity for the socket based client-server design, but more for a clean and fast portable multithreaded graphical system derived from the native OS graphical interface.
GTK!
my 2p (just a wish)
d
david casal --0+ --- d.casal@uea.ac.uk --9+ --- www.ariada.uea.ac.uk/~dcasal --)+
As i assume that only a very small fraction of users runs the kernel on another machine as the GUI i don't see a real necessity for the socket
based
client-server design, but more for a clean and fast portable
multithreaded
graphical system derived from the native OS graphical interface.
GTK!
my 2p (just a wish)
I'm afraid that GTK is not as portable as desired. I'm for wxWindows.... but that's nothing new......
T
On Thu, 17 Oct 2002, Thomas Grill wrote:
I'm afraid that GTK is not as portable as desired. I'm for wxWindows.... but that's nothing new......
Two thoughs:
A multithreaded solution might be really faster than the socket communication we have now, but it doesn't solve the problem from the root, which is, actually a communication overhead from the GUI to the engine.
Therefore I would (this is really merely theroretical, but as I see that there are several people interested in doing things with the GUI, I might as well say how I would do it, if I had the time and motivation).
First, get rid of the communication overhead. Build a clean layer between a GUI abstraction (and not an implementation how it is now) and the engine.. if desired with different ways of communication (threaded, sockets, ...)
Then we can think of porting to QT, gtk, wxwindows, whatever.
Another thing that could be done is extending the GUI on the TCL/TK side, ala "snack", for example. This would reduce overhead for expensive GUI objects, because functionality is built into the widgets.
tcl/tk allows you to add new widgets, which makes the GUI a lot nicer. I did some experiments too with that, and came to the conclusion that it can be done with tcl/tk.
Guenter
First, get rid of the communication overhead. Build a clean layer between a GUI abstraction (and not an implementation how it is now) and the engine.. if desired with different ways of communication (threaded, sockets, ...)
Then we can think of porting to QT, gtk, wxwindows, whatever.
Nothing to add. To my mind, that's the way to go.
Another thing that could be done is extending the GUI on the TCL/TK side, ala "snack", for example. This would reduce overhead for expensive GUI objects, because functionality is built into the widgets.
This would only be a solution if one wanted to stay with TCL/TK......
T
On Thu, 17 Oct 2002, Thomas Grill wrote:
Another thing that could be done is extending the GUI on the TCL/TK side, ala "snack", for example. This would reduce overhead for expensive GUI objects, because functionality is built into the widgets.
This would only be a solution if one wanted to stay with TCL/TK......
Yes, of course, ... but it would offer a smooth way for the transition.
We could write a "object" widget (edititing functionality etc, in the tk widget set) and a way how this communicates with pd. This way we already cover 95 % of all "graphical" stuff in a pd patch, and we have a proof of concept. Then we could go on like this with other graphical objects.
Well, just an idea, have to think more about it....
Guenter
Hi, guenter geiger hat gesagt: // guenter geiger wrote:
We had several discussions already about redesigning the pd<->GUI comunication, I think that this in general is starting to be a bottleneck.
I think, the GUI is indeed interfering more and more with the realtime audio. The worst it has become with the graph on parent feature. This is a very cool feature, maybe one of the most useful innovations in Pd 0.35, but when I try to move a GOP abstraction in the parent, it badly interrupts the audio stream to the dac~s.
This is not good. Of course, one could lessen this problem with a threading design, but I think in my naivity, that 'repairing' the GUI might be better/easier.
ciao
hi Thomas,
sorry about being so slow with responding (surely much less responsive than tcl/tk-based gui:)
Thomas Grill wrote:
am 16.10.2002 15:21 Uhr schrieb Krzysztof Czaja unter czaja@chopin.edu.pl:
...
actually I need only mouse _position_ polling (or, strictly speaking, sampling), which is available by calling winfo pointerxy.
Hmmm, i don't understand the advantage of polling compared to position events. If you want to poll from TCL/TK you would have to trigger a TK function and get the values via a bound message (but you know that). So using the mouse events, cache the position and do the polling (if necessary) internally seems more convenient to me.
well, this is just how MouseState works -- by default, it has to be banged in order to report mouse position (ok, this should be called sampling, not polling). Moreover, winfo pointerxy seems to be the only way of just getting the current mouse position reliably, i.e. to cover entire screen (as long as any of Pd windows is focused), and not being hampered by anybody imposing weird hooks upon pd.tk...
About the main subject of this thread -- I really like Guenter's idea of defining an abstract middle layer (or just a simple, toolkit-independent api)...
Krzysztof
-
david casal -
Frank Barknecht -
guenter geiger -
Krzysztof Czaja -
Thomas Grill