Greetings,
Is there anything in Pd comparable to MaxMSP's defer_low() mechanism? As I understand it, defer_low() will force a function to proceed at a more leisurely pace to assure that it doesn't crowd out the highest-priority DSP operations. In terms of observed performance, when the function in question was run with a regular function call in MaxMSP, it resulted in all kinds of horrible glitching in the audio playback. After running it through defer_low() everything worked quite smartly. In Pd, running the function with a regular function call simply crashes Pd after a few seconds of operation.
TIA, Eric
There is no direct correlate, but the DSP pros might be able to offer other solutions. AFAIK, Pd does not have the high and low priority DSP processing.
.hc
On Dec 5, 2006, at 2:24 PM, Eric Lyon wrote:
Greetings,
Is there anything in Pd comparable to MaxMSP's defer_low() mechanism? As I understand it, defer_low() will force a function to proceed at a more leisurely pace to assure that it doesn't crowd out the highest-priority DSP operations. In terms of observed performance, when the function in question was run with a regular function call in MaxMSP, it resulted in all kinds of horrible glitching in the audio playback. After running it through defer_low () everything worked quite smartly. In Pd, running the function with a regular function call simply crashes Pd after a few seconds of operation.
TIA, Eric
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
------------------------------------------------------------------------
¡El pueblo unido jamás será vencido!
Hi Eric, currently there's no low-priority messaging in Miller's pd version, but there is in the devel branch. It's one of the features that have been there for some time and i'm trying to provide a patch for Miller, so that it can make it into vanilla PD some time in the future. Hopefully the auto-discard mechanism of the patch tracker will blow up before that time. Don't expect the feature to be available soon.
all the best, Thomas
Am 05.12.2006 um 20:24 schrieb Eric Lyon:
Greetings,
Is there anything in Pd comparable to MaxMSP's defer_low() mechanism? As I understand it, defer_low() will force a function to proceed at a more leisurely pace to assure that it doesn't crowd out the highest-priority DSP operations. In terms of observed performance, when the function in question was run with a regular function call in MaxMSP, it resulted in all kinds of horrible glitching in the audio playback. After running it through defer_low () everything worked quite smartly. In Pd, running the function with a regular function call simply crashes Pd after a few seconds of operation.
TIA, Eric
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
On Wed, 6 Dec 2006, Thomas Grill wrote:
currently there's no low-priority messaging in Miller's pd version, but there is in the devel branch. It's one of the features that have been there for some time and i'm trying to provide a patch for Miller, so that it can make it into vanilla PD some time in the future. Hopefully the auto-discard mechanism of the patch tracker will blow up before that time. Don't expect the feature to be available soon.
So, how does one use that feature?
Is it that all the audio is high-priority and all the non-audio is low-priority?
Is it configurable by writers of externals?
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju | Freelance Digital Arts Engineer, Montréal QC Canada
On Dec 5, 2006, at 10:13 PM, Mathieu Bouchard wrote:
On Wed, 6 Dec 2006, Thomas Grill wrote:
currently there's no low-priority messaging in Miller's pd version, but there is in the devel branch. It's one of the features that have been there for some time and i'm trying to provide a patch for Miller, so that it can make it into vanilla PD some time in the future. Hopefully the auto-discard mechanism of the patch tracker will blow up before that time. Don't expect the feature to be available soon.
So, how does one use that feature?
Is it that all the audio is high-priority and all the non-audio is low-priority?
Isn't that basically the way it is now? Correct me if I am wrong, but isn't audio data processed first in each cycle, then message data?
FYI: Max since 4.3 (I believe) has two audio threads, a high priority and a low priority. "qelems" and the defer() functions give you access to the low priority thread in externals. Another detail of not is that the threads are cooperative, not pre-emptive. If a give process doesn't defer(), then it won't give up that thread until its done processing.
.hc
Is it configurable by writers of externals?
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju | Freelance Digital Arts Engineer, Montréal QC Canada_______________________________________________ PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
------------------------------------------------------------------------
Looking at things from a more basic level, you can come up with a more direct solution... It may sound small in theory, but it in practice, it can change entire economies. - Amy Smith
On Tue, 5 Dec 2006, Hans-Christoph Steiner wrote:
On Dec 5, 2006, at 10:13 PM, Mathieu Bouchard wrote:
Is it that all the audio is high-priority and all the non-audio is low-priority?
Isn't that basically the way it is now?
In Pd CANONICAL? where? where? no, there's only one thread. (apart from the watchdog and pd-gui and the threads that some externs use)
Correct me if I am wrong, but isn't audio data processed first in each cycle, then message data?
In Pd CANONICAL that's in the same thread, so it doesn't matter whether it's the first in each cycle, because a message cycle can expand and take the time allocated to 20 cycles at once. If your soundcard is running at 48000 Hz and your blocksize is 64, then your block-clock is running at 750 Hz, which is 25 times the full NTSC video framerate or 30 times the full PAL framerate; and then both GEM and GridFlow process each frame in one message cycle; I think I remember that PDP can do it differently, but I think that most people don't use that feature (?).
FYI: Max since 4.3 (I believe) has two audio threads, a high priority and a low priority. "qelems" and the defer() functions give you access to the low priority thread in externals. Another detail of not is that the threads are cooperative, not pre-emptive. If a give process doesn't defer(), then it won't give up that thread until its done processing.
Ok, that's not the kind of threads that I was thinking about. What is MAX doing to keep JITTER from causing audio JITTER ?
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju | Freelance Digital Arts Engineer, Montréal QC Canada
On 12/5/06, Mathieu Bouchard matju@artengine.ca wrote:
What is MAX doing to keep JITTER from causing audio JITTER ?
I think there's some XML involved.
On Tue, 5 Dec 2006, chris clepper wrote:
On 12/5/06, Mathieu Bouchard matju@artengine.ca wrote:
What is MAX doing to keep JITTER from causing audio JITTER ?
I think there's some XML involved.
I see, XML threads... it's easy, XML = hypertext and hypertext is kind of like hyperthreads... and then Outlook supports XML and can sort mails by thread. It all makes sense.
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju | Freelance Digital Arts Engineer, Montréal QC Canada
On Tue, Dec 05, 2006 at 11:16:54PM -0500, Mathieu Bouchard wrote:
On Tue, 5 Dec 2006, chris clepper wrote:
On 12/5/06, Mathieu Bouchard matju@artengine.ca wrote:
What is MAX doing to keep JITTER from causing audio JITTER ?
I think there's some XML involved.
I see, XML threads... it's easy, XML = hypertext and hypertext is kind of like hyperthreads... and then Outlook supports XML and can sort mails by thread. It all makes sense.
Which is why mail from Outlook bounces, because of all the dead rubber chickens in it.
Chr
------------------- chris@mccormick.cx http://mccormick.cx
I think that you guys are the reason why I get shit from the .microsound list. All your damn subtle wisecracks are rubbing off on me.
~Kyle
On 12/5/06, Chris McCormick chris@mccormick.cx wrote:
On Tue, Dec 05, 2006 at 11:16:54PM -0500, Mathieu Bouchard wrote:
On Tue, 5 Dec 2006, chris clepper wrote:
On 12/5/06, Mathieu Bouchard matju@artengine.ca wrote:
What is MAX doing to keep JITTER from causing audio JITTER ?
I think there's some XML involved.
I see, XML threads... it's easy, XML = hypertext and hypertext is kind
of
like hyperthreads... and then Outlook supports XML and can sort mails by thread. It all makes sense.
Which is why mail from Outlook bounces, because of all the dead rubber chickens in it.
Chr
chris@mccormick.cx http://mccormick.cx
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Hans-Christoph Steiner wrote:
On Dec 5, 2006, at 10:13 PM, Mathieu Bouchard wrote:
On Wed, 6 Dec 2006, Thomas Grill wrote:
currently there's no low-priority messaging in Miller's pd version, but there is in the devel branch. It's one of the features that have been there for some time and i'm trying to provide a patch for Miller, so that it can make it into vanilla PD some time in the future. Hopefully the auto-discard mechanism of the patch tracker will blow up before that time. Don't expect the feature to be available soon.
So, how does one use that feature?
Is it that all the audio is high-priority and all the non-audio is low-priority?
Isn't that basically the way it is now? Correct me if I am wrong, but isn't audio data processed first in each cycle, then message data?
just think: - if you have a lot of audio calculations going on, does the message engine stutter? - if you have a lot of message calculations going on, does the audio engine stutter?
so which part of pd is higher prioritized: messages or audio?
and oh, you can have priorities even without threads...
fmga.sdr IOhannes
Georg Holzmann wrote:
Hallo!
and oh, you can have priorities even without threads...
and how?
Do you mean a bigger blocksize ... ?
seems i have been unclear here: i was making a general statement about priorities and threads (not related to pd at all); the concept of priorities does not necessarily mean, that all tasks are guaranteed to be fully executed; you could just discard the todo-list of a low-priority task when you are running out of time.
for example, pd does have 2 priorities (audio and messages), even though they are in 1 thread.
i hope this is clearer.
mfg.adsr IOhannes
Am 06.12.2006 um 04:21 schrieb Hans-Christoph Steiner:
On Dec 5, 2006, at 10:13 PM, Mathieu Bouchard wrote:
On Wed, 6 Dec 2006, Thomas Grill wrote:
currently there's no low-priority messaging in Miller's pd version, but there is in the devel branch. It's one of the features that have been there for some time and i'm trying to provide a patch for Miller, so that it can make it into vanilla PD some time in the future. Hopefully the auto-discard mechanism of the patch tracker will blow up before that time. Don't expect the feature to be available soon.
So, how does one use that feature?
Is it that all the audio is high-priority and all the non-audio is low-priority?
Isn't that basically the way it is now? Correct me if I am wrong, but isn't audio data processed first in each cycle, then message data?
The order is like that, but if you have too much DSP operation you will get clicking, and vice versa, when you have too much message operation, the DSP tick can't be done in time, underflowing your DSP output ring buffer.
FYI: Max since 4.3 (I believe) has two audio threads, a high priority and a low priority. "qelems" and the defer() functions give you access to the low priority thread in externals. Another detail of not is that the threads are cooperative, not pre- emptive. If a give process doesn't defer(), then it won't give up that thread until its done processing.
I'm not sure about that cooperativity. I'm not aware that an external must do a defer so that another low-priority operation can be done. I'd rather think that DSP, high and low priority threads are 3 threads with different priority that share a thread lock, so that only one can be active at a time (but i could also be wrong)
Low priority messaging works in a way that output messages (like through an outlet or send) do not directly call a function handler receiving that message (as it currently is in PD), but are rather queued. The queue is processed and call the proper handler whenever higher priority threads leave time for it.
In pd-devel there's a (thread-safe) sys_callback API function which installs an idle time callback. These callbacks can decide to run once or multiple times, depending on their return value. The idle callback queue is processed by the scheduler whenever DSP and normal message processing leaves time to do so. This works very well, but only if the idle functions don't consume a lot of time. If a low-priority thread consume more time, a secondary thread must be used, but idle callbacks can be used to communicate with the main pd thread.
The problem for inclusion into pd-miller is that a small infrastructure of atomic operations has to be included, which requires some assembler code for generic OS support (though functions are there for linux, Windows and OSX >= 10.4), which might frighten Miller, as in the past with SIMD.
all the best, Thomas
Low priority messaging works in a way that output messages (like through an outlet or send) do not directly call a function handler receiving that message (as it currently is in PD), but are rather queued. The queue is processed and call the proper handler whenever higher priority threads leave time for it.
sorry, that should read:
.... whenever DSP and higher priority messaging leave time for it
On Dec 6, 2006, at 7:55 AM, Thomas Grill wrote:
Am 06.12.2006 um 04:21 schrieb Hans-Christoph Steiner:
On Dec 5, 2006, at 10:13 PM, Mathieu Bouchard wrote:
On Wed, 6 Dec 2006, Thomas Grill wrote:
currently there's no low-priority messaging in Miller's pd version, but there is in the devel branch. It's one of the features that have been there for some time and i'm trying to provide a patch for Miller, so that it can make it into vanilla PD some time in the future. Hopefully the auto-discard mechanism of the patch tracker will blow up before that time. Don't expect the feature to be available soon.
So, how does one use that feature?
Is it that all the audio is high-priority and all the non-audio is low-priority?
Isn't that basically the way it is now? Correct me if I am wrong, but isn't audio data processed first in each cycle, then message data?
The order is like that, but if you have too much DSP operation you will get clicking, and vice versa, when you have too much message operation, the DSP tick can't be done in time, underflowing your DSP output ring buffer.
FYI: Max since 4.3 (I believe) has two audio threads, a high priority and a low priority. "qelems" and the defer() functions give you access to the low priority thread in externals. Another detail of not is that the threads are cooperative, not pre- emptive. If a give process doesn't defer(), then it won't give up that thread until its done processing.
I'm not sure about that cooperativity. I'm not aware that an external must do a defer so that another low-priority operation can be done. I'd rather think that DSP, high and low priority threads are 3 threads with different priority that share a thread lock, so that only one can be active at a time (but i could also be wrong)
Low priority messaging works in a way that output messages (like through an outlet or send) do not directly call a function handler receiving that message (as it currently is in PD), but are rather queued. The queue is processed and call the proper handler whenever higher priority threads leave time for it.
Do you know any good resources on Max's threads? I can't find anything. They seem to be poorly documented in terms of how they are actually implemented. They probably aren't OS threads or pthreads since they have tightly controlled timing.
I've only found this, which sheds a little light:
http://www.cycling74.com/story/2005/5/2/133649/9742
I think I misunderstood the defer() stuff. It seems to be optional. So basically Max/MSP gives you two queues to put your events on: high and low.
The timing bugs that you run into with the "Overdrive" threads turned on must be vicious to debug, since the Max language was designed to be deterministic, but the threading throws that out the window.
.hc
In pd-devel there's a (thread-safe) sys_callback API function which installs an idle time callback. These callbacks can decide to run once or multiple times, depending on their return value. The idle callback queue is processed by the scheduler whenever DSP and normal message processing leaves time to do so. This works very well, but only if the idle functions don't consume a lot of time. If a low-priority thread consume more time, a secondary thread must be used, but idle callbacks can be used to communicate with the main pd thread.
The problem for inclusion into pd-miller is that a small infrastructure of atomic operations has to be included, which requires some assembler code for generic OS support (though functions are there for linux, Windows and OSX >= 10.4), which might frighten Miller, as in the past with SIMD.
------------------------------------------------------------------------
I spent 33 years and four months in active military service and during that period I spent most of my time as a high class muscle man for Big Business, for Wall Street and the bankers. - General Smedley Butler
FYI: There is no auto-discard feature on the tracker. The only way to delete items is to manually do it. The "Pending" status automatically switches the tracker item to "Closed", which is definitely not deleted. Everything is still there.
This is just a tool to keep the tracker lists manageable. If a patch has an question from Miller that was not answered for over one year, I think its appropriate if its closed. Then anyone can re-open it and answer the question at any time.
If that list just gets longer and longer, then its less likely to be used.
.hc
On Dec 5, 2006, at 8:24 PM, Thomas Grill wrote:
Hi Eric, currently there's no low-priority messaging in Miller's pd version, but there is in the devel branch. It's one of the features that have been there for some time and i'm trying to provide a patch for Miller, so that it can make it into vanilla PD some time in the future. Hopefully the auto-discard mechanism of the patch tracker will blow up before that time. Don't expect the feature to be available soon.
all the best, Thomas
Am 05.12.2006 um 20:24 schrieb Eric Lyon:
Greetings,
Is there anything in Pd comparable to MaxMSP's defer_low() mechanism? As I understand it, defer_low() will force a function to proceed at a more leisurely pace to assure that it doesn't crowd out the highest-priority DSP operations. In terms of observed performance, when the function in question was run with a regular function call in MaxMSP, it resulted in all kinds of horrible glitching in the audio playback. After running it through defer_low() everything worked quite smartly. In Pd, running the function with a regular function call simply crashes Pd after a few seconds of operation.
TIA, Eric
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
------------------------------------------------------------------------
"[W]e have invented the technology to eliminate scarcity, but we are deliberately throwing it away to benefit those who profit from scarcity." -John Gilmore
Hallo!
currently there's no low-priority messaging in Miller's pd version, but there is in the devel branch. It's one of the features that have been there for some time and i'm trying to provide a patch for Miller, so that it can make it into vanilla PD some time in the future. Hopefully the auto-discard mechanism of the patch tracker will blow up before that time. Don't expect the feature to be available soon.
hope that you get it in soon ... ;)
however, I tried to use this functionality af pd-devel for vanilla pd and used these features as an external, you can see some examples here: http://grh.mur.at/software/threadlib.html
LG Georg
-
chris clepper -
Chris McCormick -
Eric Lyon -
Georg Holzmann -
Hans-Christoph Steiner -
IOhannes m zmoelnig -
Kyle Klipowicz -
Mathieu Bouchard -
Thomas Grill