Hi all, there has been some interest in the pyext object lately which doesn't work with vanilla pd very well. This seems to be because PD's built-in stack checking can't cope with PD functions called from a different thread. Following is the diff for *** src/m_obj.c of PD 0.37-1 *** for a different stack checking approach - it's a simple iteration count which should be far more portable. I'm not sure to which value to set the maximum iteration count - it's 1000 for now. Please note, the devel_0_37 branch already has these changes.
best greetings, Thomas
11a12,15
/* T.Grill - define for a modified, more portable method to detect stack
overflows */
#define NEWSTACKMETH
258a263,267
#ifdef NEWSTACKMETH /* T.Grill - count iterations rather than watch the stack pointer */ static int stackcount = 0; /* iteration counter */ #define STACKITER 1000 /* maximum iterations allowed */ #else
260a270,271
#endif
265a277
267a280
#ifndef NEWSTACKMETH
270a284
#endif
312a327
#ifndef NEWSTACKMETH
313a329
#endif
319c335,338 < char c; ---
#ifdef NEWSTACKMETH if(++stackcount >= STACKITER) #else char c;
320a340
#endif
323a344,346
#ifdef NEWSTACKMETH --stackcount; #endif
330c353,356 < char c; ---
#ifdef NEWSTACKMETH if(++stackcount >= STACKITER) #else char c;
331a358
#endif
345a373,375
#ifdef NEWSTACKMETH --stackcount; #endif
351c381,384 < char c; ---
#ifdef NEWSTACKMETH if(++stackcount >= STACKITER) #else char c;
352a386
#endif
355a390,392
#ifdef NEWSTACKMETH --stackcount; #endif
361c398,401 < char c; ---
#ifdef NEWSTACKMETH if(++stackcount >= STACKITER) #else char c;
362a403
#endif
365a407,409
#ifdef NEWSTACKMETH --stackcount; #endif
371c415,418 < char c; ---
#ifdef NEWSTACKMETH if(++stackcount >= STACKITER) #else char c;
372a420
#endif
375a424,426
#ifdef NEWSTACKMETH --stackcount; #endif
381c432,435 < char c; ---
#ifdef NEWSTACKMETH if(++stackcount >= STACKITER) #else char c;
382a437
#endif
385a441,443
#ifdef NEWSTACKMETH --stackcount; #endif
hi Thomas,
I wonder, what is your opinion about other threads calling outlet functions -- how reliable it is, given that an outlet function in turn calls a method of an unknown class, possibly not thread-safe at all?
Btw, do I read well, that there are two dispatching methods implemented in flext -- one uses the main thread, and the clock_set() as the trigger, while the other is background dispatcher, and that one is apparently default for Pd? Is the background method a much better choice, empirically?
I am asking, because some time ago I have been thinking about introducing optionally threaded interpreters into the `plustot' library (+tot loads tcl interpreter(s) into the pd process, unlike tot, which uses the pd-gui process). Due to design troubles, I put the threading feature off for better times.
Krzysztof
Thomas Grill wrote: ...
functions called from a different thread.
Hi Krzysztof,
I wonder, what is your opinion about other threads calling outlet functions -- how reliable it is, given that an outlet function in turn calls a method of an unknown class, possibly not thread-safe at all?
There are the sys_lock and sys_unlock API functions that guarantee that a locked zone can only be active when PD is idle (is microsleep). Call to the PD API may only be made in locked zones, therefore other externals need not be threadsafe.
Btw, do I read well, that there are two dispatching methods implemented in flext -- one uses the main thread, and the clock_set() as the trigger, while the other is background dispatcher, and that one is apparently default for Pd? Is the background method a much better choice, empirically?
In flext there's a message queue for all messages from non-PD threads or for explicitly queued ones. It will only be emptied when the sys_lock is set.
sorry, in a hurry, Thomas
thanks Thomas,
Thomas Grill wrote: ...
There are the sys_lock and sys_unlock API functions that guarantee that a locked zone can only be active when PD is idle (is microsleep).
ok, I should have done my homework first... still, I am not sure the lock/unlock calls I can see in my copy of Pd scheduler are not misplaced. I think, all the external messages, like gui events (mouse and keyboard), net receivers, pipes, etc., are being polled into Pd via the microsleep routine, so they seem not to be locked out by the other thread?
...
In flext there's a message queue for all messages from non-PD threads or for
Forgive me, but I still do not understand it fully. One thing, simple and robust, is queuing messages, and arming a clock, for dispatching them in the main thread. However, if we are to distribute message passing among different threads anyway, what is to be gained, in the real world, by using the background dispatcher, instead of each thread locking Pd when issuing its message?
Krzysztof
Hi Krzysztof,
There are the sys_lock and sys_unlock API functions that guarantee that
a
locked zone can only be active when PD is idle (is microsleep).
ok, I should have done my homework first... still, I am not sure the lock/unlock calls I can see in my copy of Pd scheduler are not misplaced. I think, all the external messages, like gui events (mouse and keyboard), net receivers, pipes, etc., are being polled into Pd via the microsleep routine, so they seem not to be locked out by the other thread?
Hmmm, i think you are right, but the question is if these can really collide with the threads used by PD externals. I think i remember that it was Miller's suggestion to place them there and i hope that my lack of insight into the PD internals didn't mix things up here. It's a good thing to reconsider the whole construction since threads are used more and more these days.
In flext there's a message queue for all messages from non-PD threads or
for
Forgive me, but I still do not understand it fully. One thing, simple and robust, is queuing messages, and arming a clock, for dispatching them in the main thread. However, if we are to distribute message passing among different threads anyway, what is to be gained, in the real world, by using the background dispatcher, instead of each thread locking Pd when issuing its message?
The gain is that the routines using the message don't need to wait for the thread lock to open. I have to admit on the other hand that i was lazy doing profiling... that's what still has to be done. Since practically all important features are readily implemented, maybe it's the time to do so now (after the summer vacation).
best greetings, Thomas
-
Krzysztof Czaja -
Thomas Grill