Hello all,
I have skimmed Miller's paper from Pd-con about [pd~], and it looks like it has potential for taking advantage of multiple-core CPUs. I need to read it in a little more detail to digest it fully, but I'm wondering (and this is directed mostly at Frank B.): could [polypoly] and/or [nqpoly] use [pd~] for each voice/replicated-patch instance?
Phil Stone www.pkstonemusic.com
It would definitely be possible to write a pdpoly~ but usually it
would be easier to manually manage 2-4 instances. Few people have
more than 4 cores. I see those poly objects as useful for 10+ and
make managing 100s or 1000s possible.
.hc
On Sep 8, 2009, at 1:24 PM, Phil Stone wrote:
Hello all,
I have skimmed Miller's paper from Pd-con about [pd~], and it looks
like it has potential for taking advantage of multiple-core CPUs. I
need to read it in a little more detail to digest it fully, but I'm
wondering (and this is directed mostly at Frank B.): could
[polypoly] and/or [nqpoly] use [pd~] for each voice/replicated-patch
instance?Phil Stone www.pkstonemusic.com
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Using ReBirth is like trying to play an 808 with a long stick. - David Zicarelli
Hi Hans,
Thanks for replying. I don't quite understand what you mean by "manually manage". As far as I know, without something like [pd~], there's no way to divide up and assign the Pd audio process to more than one core. Half of the cores on a quad-core are therefore useless to Pd (accounting for the fact that the graphical process gets its own core).
Phil
Hans-Christoph Steiner wrote:
It would definitely be possible to write a pdpoly~ but usually it would be easier to manually manage 2-4 instances. Few people have more than 4 cores. I see those poly objects as useful for 10+ and make managing 100s or 1000s possible.
.hc
On Sep 8, 2009, at 1:24 PM, Phil Stone wrote:
Hello all,
I have skimmed Miller's paper from Pd-con about [pd~], and it looks like it has potential for taking advantage of multiple-core CPUs. I need to read it in a little more detail to digest it fully, but I'm wondering (and this is directed mostly at Frank B.): could [polypoly] and/or [nqpoly] use [pd~] for each voice/replicated-patch instance?
Phil Stone www.pkstonemusic.com
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Using ReBirth is like trying to play an 808 with a long stick.
-David Zicarelli
Phil Stone wrote:
Hi Hans,
Thanks for replying. I don't quite understand what you mean by "manually manage". As far as I know, without something like [pd~], there's no way to divide up and assign the Pd audio process to more than one core. Half of the cores on a quad-core are therefore useless to Pd (accounting for the fact that the graphical process gets its own core).
the problem is, that poly-class objects are usually meant for _many_ objects (10+; i'm only repeating here what hans has already said). [pd~] will fork a new thread for each of it's instances. when doing multi-core processing (and this is really the only thing pd~ is good for; e.g .it's not good if you want to have different priorities / asynchronous processing), you usually don't want to create more threads than you have cores. why? performance reasons! if you create e.g. 1000 threads for 1000 instances of [doodle~] on a quad-core machine, your computer will spend more time handling context-switches and the like (that is: the overhead for managing the threads) than doing the actual job. as a rule of thumb, the optimum number of threads is about the number of cores you want to use.
since what is sold to customers as "multi-core" processors usually does not involve more than 4 cores, the "best" (though probably not the most comfortable) way to assign work to the cores from within Pd is doing it "manually"
fmgasdr.# IOhannes
Sorry if this question is obvious, may be an alternate for live audio processing with clusters: does it exist some netsend/netreceive for audio in Puredata ? I remember having using one (experimental) few years ago but was within MaxMSP...
fred
IOhannes m zmoelnig wrote:
Phil Stone wrote:
Hi Hans,
Thanks for replying. I don't quite understand what you mean by "manually manage". As far as I know, without something like [pd~], there's no way to divide up and assign the Pd audio process to more than one core. Half of the cores on a quad-core are therefore useless to Pd (accounting for the fact that the graphical process gets its own core).
the problem is, that poly-class objects are usually meant for _many_ objects (10+; i'm only repeating here what hans has already said). [pd~] will fork a new thread for each of it's instances. when doing multi-core processing (and this is really the only thing pd~ is good for; e.g .it's not good if you want to have different priorities / asynchronous processing), you usually don't want to create more threads than you have cores. why? performance reasons! if you create e.g. 1000 threads for 1000 instances of [doodle~] on a quad-core machine, your computer will spend more time handling context-switches and the like (that is: the overhead for managing the threads) than doing the actual job. as a rule of thumb, the optimum number of threads is about the number of cores you want to use.
since what is sold to customers as "multi-core" processors usually does not involve more than 4 cores, the "best" (though probably not the most comfortable) way to assign work to the cores from within Pd is doing it "manually"
fmgasdr.# IOhannes
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
did someone succeeded in making x86 pd_darwin or ppc pd_linux for netsend~/netreceive~ ? or is there something newer than http://www.nullmedium.de/dev/netsend~/ ?
thank's for your help, fred
fred-ordi wrote:
Sorry if this question is obvious, may be an alternate for live audio processing with clusters: does it exist some netsend/netreceive for audio in Puredata ? I remember having using one (experimental) few years ago but was within MaxMSP...
fred
IOhannes m zmoelnig wrote:
Phil Stone wrote:
Hi Hans,
Thanks for replying. I don't quite understand what you mean by "manually manage". As far as I know, without something like [pd~], there's no way to divide up and assign the Pd audio process to more than one core. Half of the cores on a quad-core are therefore useless to Pd (accounting for the fact that the graphical process gets its own core).
the problem is, that poly-class objects are usually meant for _many_ objects (10+; i'm only repeating here what hans has already said). [pd~] will fork a new thread for each of it's instances. when doing multi-core processing (and this is really the only thing pd~ is good for; e.g .it's not good if you want to have different priorities / asynchronous processing), you usually don't want to create more threads than you have cores. why? performance reasons! if you create e.g. 1000 threads for 1000 instances of [doodle~] on a quad-core machine, your computer will spend more time handling context-switches and the like (that is: the overhead for managing the threads) than doing the actual job. as a rule of thumb, the optimum number of threads is about the number of cores you want to use.
since what is sold to customers as "multi-core" processors usually does not involve more than 4 cores, the "best" (though probably not the most comfortable) way to assign work to the cores from within Pd is doing it "manually"
fmgasdr.# IOhannes
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
There's a recent update here: http://www.remu.fr/sound-delta/netsend~/?page_id=7 Nick soundsorange.net
did someone succeeded in making x86 pd_darwin or ppc pd_linux for netsend~/netreceive~ ? or is there something newer than http://www.nullmedium.de/dev/netsend~/ ?
thank's for your help, fred
fred-ordi wrote:
Sorry if this question is obvious, may be an alternate for live audio processing with clusters: does it exist some netsend/netreceive for audio in Puredata ? I remember having using one (experimental) few years ago but was within MaxMSP...
fred
IOhannes m zmoelnig wrote:
Phil Stone wrote:
Hi Hans,
Thanks for replying. I don't quite understand what you mean by "manually manage". As far as I know, without something like [pd~], there's no way to divide up and assign the Pd audio process to more than one core. Half of the cores on a quad-core are therefore useless to Pd (accounting for the fact that the graphical process gets its own core).
the problem is, that poly-class objects are usually meant for _many_ objects (10+; i'm only repeating here what hans has already said). [pd~] will fork a new thread for each of it's instances. when doing multi-core processing (and this is really the only thing pd~ is good for; e.g .it's not good if you want to have different priorities / asynchronous processing), you usually don't want to create more threads than you have cores. why? performance reasons! if you create e.g. 1000 threads for 1000 instances of [doodle~] on a quad-core machine, your computer will spend more time handling context-switches and the like (that is: the overhead for managing the threads) than doing the actual job. as a rule of thumb, the optimum number of threads is about the number of cores you want to use.
since what is sold to customers as "multi-core" processors usually does not involve more than 4 cores, the "best" (though probably not the most comfortable) way to assign work to the cores from within Pd is doing it "manually"
fmgasdr.# IOhannes
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Networking has a lot of latency and jitter, so not so good for
realtime audio. As for manually managing pd~, I mean just manually
creating as many pd~ instances as you have cores.
.hc
On Sep 9, 2009, at 5:06 AM, fred-ordi wrote:
Sorry if this question is obvious, may be an alternate for live
audio processing with clusters: does it exist some netsend/netreceive for audio in Puredata ? I remember having using one (experimental) few years ago but was
within MaxMSP...fred
IOhannes m zmoelnig wrote:
Phil Stone wrote:
Hi Hans,
Thanks for replying. I don't quite understand what you mean by
"manually manage". As far as I know, without something like
[pd~], there's no way to divide up and assign the Pd audio process
to more than one core. Half of the cores on a quad-core are
therefore useless to Pd (accounting for the fact that the
graphical process gets its own core).the problem is, that poly-class objects are usually meant for
_many_ objects (10+; i'm only repeating here what hans has already
said). [pd~] will fork a new thread for each of it's instances. when doing multi-core processing (and this is really the only thing
pd~ is good for; e.g .it's not good if you want to have different
priorities / asynchronous processing), you usually don't want to
create more threads than you have cores. why? performance reasons! if you create e.g. 1000 threads for 1000
instances of [doodle~] on a quad-core machine, your computer will
spend more time handling context-switches and the like (that is:
the overhead for managing the threads) than doing the actual job. as a rule of thumb, the optimum number of threads is about the
number of cores you want to use. since what is sold to customers as "multi-core" processors usually
does not involve more than 4 cores, the "best" (though probably not
the most comfortable) way to assign work to the cores from within
Pd is doing it "manually" fmgasdr.# IOhannes
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
I have the audacity to believe that peoples everywhere can have three
meals a day for their bodies, education and culture for their minds,
and dignity, equality and freedom for their spirits. - Martin
Luther King, Jr.
Thanks for clarifying that, Hans, and for pointing out the issue with threads, IOhannes. One shouldn't be profligate with [pd~]s, strewing them all about and expecting performance gains -- therefore, one [pd~] per voice instance in a [polypoly] patch is probably not a good idea until we have 64-core CPUs as a regular thing! :-).
However, with judicious use, [pd~] seems like it will allow Pd to scale to future processor design, and that's a good thing.
Thanks again,
Phil
Hans-Christoph Steiner wrote:
[snip] As for manually managing pd~, I mean just manually creating as many pd~ instances as you have cores.
.hc
IOhannes m zmoelnig wrote:
Phil Stone wrote:
Hi Hans,
Thanks for replying. I don't quite understand what you mean by "manually manage". As far as I know, without something like [pd~], there's no way to divide up and assign the Pd audio process to more than one core. Half of the cores on a quad-core are therefore useless to Pd (accounting for the fact that the graphical process gets its own core).
the problem is, that poly-class objects are usually meant for _many_ objects (10+; i'm only repeating here what hans has already said). [pd~] will fork a new thread for each of it's instances. when doing multi-core processing (and this is really the only thing pd~ is good for; e.g .it's not good if you want to have different priorities / asynchronous processing), you usually don't want to create more threads than you have cores. why? performance reasons! if you create e.g. 1000 threads for 1000 instances of [doodle~] on a quad-core machine, your computer will spend more time handling context-switches and the like (that is: the overhead for managing the threads) than doing the actual job. as a rule of thumb, the optimum number of threads is about the number of cores you want to use. since what is sold to customers as "multi-core" processors usually does not involve more than 4 cores, the "best" (though probably not the most comfortable) way to assign work to the cores from within Pd is doing it "manually" fmgasdr.# IOhannes
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
I have the audacity to believe that peoples everywhere can have three meals a day for their bodies, education and culture for their minds, and dignity, equality and freedom for their spirits. - Martin Luther King, Jr.
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
On Wed, 9 Sep 2009, Phil Stone wrote:
Thanks for clarifying that, Hans, and for pointing out the issue with threads, IOhannes. One shouldn't be profligate with [pd~]s, strewing them all about and expecting performance gains -- therefore, one [pd~] per voice instance in a [polypoly] patch is probably not a good idea until we have 64-core CPUs as a regular thing! :-). However, with judicious use, [pd~] seems like it will allow Pd to scale to future processor design, and that's a good thing.
For large numbers of CPUs, and even for not so large ones, [pd~] is not so useful, as it has to be put explicitly in places where one would rather not have to put it, and where it can be quite complicated to introduce it.
If [pd~] were more like [pd] it would be more transparent; it would be easier to switch between the two. However, the most useful load-balancing cutting lines are not necessarily those of subpatches and they're even less those of whole patches (thus you have to artificially create separate patches wherever you want to spread work on several CPUs). I would believe that the most useful solutions would look more like Blechmann's [detach] and [join], but I don't know all of the implications of it.
...
There is also a naming problem. It's expected that a ~ sign contrast such as [exp~] vs [exp] means that both classes are as similar as possible except one works on signals and the other doesn't. Despite Pd having some quirks about this, [pd~] introduces a big mental clash, so that now, when one explains the general meaning of ~, on has to make an exception for [pd~]. (If [pd~] were more like [pd], this problem wouldn't be any different.)
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard, Montréal, Québec. téléphone: +1.514.383.3801
-
fred-ordi -
Hans-Christoph Steiner -
IOhannes m zmoelnig -
Mathieu Bouchard -
Nicholas Mariette -
Phil Stone