I'm planning to release our realtime convolution engine (extracted from http://meatscience.net/pages/convolution-reverb) as a GPLed Pd external.
It currently accepts 4-channel ('true stereo'), two channel or mono impulse responses, with stereo or mono output. Performance is excellent if you have SSE3 and has a fallback in case you don't, and it aims for accuracy (basically that means multi-stage scaling to keep floats within healthy sizes).
- I'd love to swipe the convolve~ external name, currently installed
by mjlib as part of pd-extended. convolve~ from mjlib appears to be a copy of pin~ ? so I think it could be taken? Maybe I mis-read the code. I've cc'ed mark who can probably clarify.
- Anyone have requests for features/api? Its currently simplistic:
- takes a "read FILENAME" message, loads the file, does a test
convolution against pink noise to normalize the gain to something sane
- caches the last N impulse responses, as the test convolution
takes a little time
- allows setting the cache size with a "cachesize N" message
- disable normalization with "normalize 0" or "normalize 1"
Features I'm considering (let me know if they sound useful): - load from an array instead of from disk (no gain normalization?) - It wouldn't be hard to enable MxN convolution if that floats somebody's boat.
I can compile/test on Mac & Linux, anyone up for helping me with Windows?
Would this be of interest for Pd-extended?
I'd love to build a granular convolution engine.... takes two
real-time signals, and extracts grains from one to convolve against the other. Anyone have ideas about this?
thanks all,
-Seth
Seth Nickell wrote:
I'm planning to release our realtime convolution engine (extracted from http://meatscience.net/pages/convolution-reverb) as a GPLed Pd external.
This is a good idea!
It currently accepts 4-channel ('true stereo'), two channel or mono impulse responses, with stereo or mono output. Performance is
What is 'true stereo' with four channels by the way?
excellent if you have SSE3 and has a fallback in case you don't, and it aims for accuracy (basically that means multi-stage scaling to keep floats within healthy sizes).
- I'd love to swipe the convolve~ external name, currently installed
by mjlib as part of pd-extended. convolve~ from mjlib appears to be a copy of pin~ ? so I think it could be taken? Maybe I mis-read the code. I've cc'ed mark who can probably clarify.
- Anyone have requests for features/api? Its currently simplistic:
- takes a "read FILENAME" message, loads the file, does a test
convolution against pink noise to normalize the gain to something sane
Is this done within the main Pd audio thread?
- caches the last N impulse responses, as the test convolution
takes a little time
- allows setting the cache size with a "cachesize N" message
To make sure I understood this: cachesize is not the size of the first partition of the partitioned convolution, but the cache that tries to avoid audio dropouts when performing the test convolution?
- disable normalization with "normalize 0" or "normalize 1"
Yes, disabling this could be a good idea! You could also add a "gain 0-1" message for manual control.
Features I'm considering (let me know if they sound useful): - load from an array instead of from disk (no gain normalization?)
Very good.
- It wouldn't be hard to enable MxN convolution if that floatssomebody's boat.
I am sure if you come up with a convolution as efficient and flexible as jconv by Fons within Pd, then soon a multichannel use and hence request will come up fast.
[...]
- Would this be of interest for Pd-extended?
This would be of interest for all Pd users, no matter if they like their externals included in a distribution of Pd ('extended') or manuall adding them to their vanilla Pd.
best, P
- Anyone have requests for features/api? Its currently simplistic:
- takes a "read FILENAME" message, loads the file, does a test convolution against pink noise to normalize the gain to something sane
Is this done within the main Pd audio thread?
The convolution engine has support for doing it either on the calling thread, or a background thread. I'm thinking of default to a background thread. That seem like the right move?
- caches the last N impulse responses, as the test convolution takes a little time - allows setting the cache size with a "cachesize N" message
To make sure I understood this: cachesize is not the size of the first partition of the partitioned convolution, but the cache that tries to avoid audio dropouts when performing the test convolution?
The convolution engine can swap-in a pre-loaded ('cached') IR in realtime without glitching... but it means keeping 2x the Impulse Response data in RAM. To keep the default API simple but useful, I'm defaulting to caching only the last 5 impulse responses in RAM. "cachesize N" lets you increase that number.... lets say in a performance you wanted to use 30 different impulse responses and you have 2GB of ram... should be nbd.
- disable normalization with "normalize 0" or "normalize 1"
Yes, disabling this could be a good idea! You could also add a "gain 0-1" message for manual control.
Its worth noting that impulse responses are usually whack without gain normalization.... like factors of hundreds to millions off a usable signal.
Features I'm considering (let me know if they sound useful): - load from an array instead of from disk (no gain normalization?)
Very good.
- It wouldn't be hard to enable MxN convolution if that floats somebody's boat.
I am sure if you come up with a convolution as efficient and flexible as jconv by Fons within Pd, then soon a multichannel use and hence request will come up fast.
I'd be interested in what flexibility means in this context, it might give me some good ideas for features to add. Efficiency-wise, last time I benchmarked its more efficient than jconv, but the difference is offset by less graceful degradation under CPU load (I convolve in background threads to preserve realtime in the main thread while avoiding an irritating patent that's going to expire soon...).
WRT to Pd's audio scheduling... are Pd signal externals held to realtime or can my dsp call vary the number of cycles it takes by 100% from call to call? VST seems to do ok with this, but AudioUnits get scheduled to run at the very last instant they possibly could. If Pd can have some variance, I can drop the threads and improve the external's degradation under high CPU load.
thanks for the feedback (also, is the best list for this kind of feedback?),
-Seth
Another question on similar lines...
Are the DSP calls liable to vary t_signal->s_n (block size) without notification? 64 samples, apparently the default on pd-extended, is doable without buffering for partitioned convolution on a modern computer, but it exacts a pretty high CPU toll, and if I have to handle random blocksize changes, it gets more expensive.
Also, since convolution is much more efficient around block sizes of 256 or 512, perhaps I should default to one of these, buffer a little, and have a "runatpdblocksize" message or somesuch?
On Mon, Apr 4, 2011 at 7:48 PM, Seth Nickell seth@meatscience.net wrote:
- Anyone have requests for features/api? Its currently simplistic:
- takes a "read FILENAME" message, loads the file, does a test convolution against pink noise to normalize the gain to something sane
Is this done within the main Pd audio thread?
The convolution engine has support for doing it either on the calling thread, or a background thread. I'm thinking of default to a background thread. That seem like the right move?
- caches the last N impulse responses, as the test convolution takes a little time - allows setting the cache size with a "cachesize N" message
To make sure I understood this: cachesize is not the size of the first partition of the partitioned convolution, but the cache that tries to avoid audio dropouts when performing the test convolution?
The convolution engine can swap-in a pre-loaded ('cached') IR in realtime without glitching... but it means keeping 2x the Impulse Response data in RAM. To keep the default API simple but useful, I'm defaulting to caching only the last 5 impulse responses in RAM. "cachesize N" lets you increase that number.... lets say in a performance you wanted to use 30 different impulse responses and you have 2GB of ram... should be nbd.
- disable normalization with "normalize 0" or "normalize 1"
Yes, disabling this could be a good idea! You could also add a "gain 0-1" message for manual control.
Its worth noting that impulse responses are usually whack without gain normalization.... like factors of hundreds to millions off a usable signal.
Features I'm considering (let me know if they sound useful): - load from an array instead of from disk (no gain normalization?)
Very good.
- It wouldn't be hard to enable MxN convolution if that floats somebody's boat.
I am sure if you come up with a convolution as efficient and flexible as jconv by Fons within Pd, then soon a multichannel use and hence request will come up fast.
I'd be interested in what flexibility means in this context, it might give me some good ideas for features to add. Efficiency-wise, last time I benchmarked its more efficient than jconv, but the difference is offset by less graceful degradation under CPU load (I convolve in background threads to preserve realtime in the main thread while avoiding an irritating patent that's going to expire soon...).
WRT to Pd's audio scheduling... are Pd signal externals held to realtime or can my dsp call vary the number of cycles it takes by 100% from call to call? VST seems to do ok with this, but AudioUnits get scheduled to run at the very last instant they possibly could. If Pd can have some variance, I can drop the threads and improve the external's degradation under high CPU load.
thanks for the feedback (also, is the best list for this kind of feedback?),
-Seth
Dear Seth,
Seth Nickell wrote:
Another question on similar lines...
Are the DSP calls liable to vary t_signal->s_n (block size) without notification? 64 samples, apparently the default on pd-extended, is doable without buffering for partitioned convolution on a modern computer, but it exacts a pretty high CPU toll, and if I have to handle random blocksize changes, it gets more expensive.
They cannot vary by themselves, but what is usually done (e.g. with FFTs), is to place an signal (tilde ~) object in a subpatch and resize the blocksize for that blocksize using the [switch~] or [block~] objects. You might consider using this very approach.
Also, since convolution is much more efficient around block sizes of 256 or 512, perhaps I should default to one of these, buffer a little, and have a "runatpdblocksize" message or somesuch?
I still have not understood if/how the user can set the duration of the first partition of you partitioned convolution, and how these partitions are structured in their (possibly increasing) sizes. Since this first paramter will define the latency-vs-CPU tradeoff it should not be preset by the developers.
P.
PS: Pd and Pd-extended use the same core, audio engine. You might want to consider Pd-extended as vanilla Pd with a folder full of precompiled externals.
On Mon, Apr 4, 2011 at 7:48 PM, Seth Nickell seth@meatscience.net wrote:
- Anyone have requests for features/api? Its currently simplistic:
- takes a "read FILENAME" message, loads the file, does a test
convolution against pink noise to normalize the gain to something sane
Is this done within the main Pd audio thread?
The convolution engine has support for doing it either on the calling thread, or a background thread. I'm thinking of default to a background thread. That seem like the right move?
- caches the last N impulse responses, as the test convolution
takes a little time
- allows setting the cache size with a "cachesize N" message
To make sure I understood this: cachesize is not the size of the first partition of the partitioned convolution, but the cache that tries to avoid audio dropouts when performing the test convolution?
The convolution engine can swap-in a pre-loaded ('cached') IR in realtime without glitching... but it means keeping 2x the Impulse Response data in RAM. To keep the default API simple but useful, I'm defaulting to caching only the last 5 impulse responses in RAM. "cachesize N" lets you increase that number.... lets say in a performance you wanted to use 30 different impulse responses and you have 2GB of ram... should be nbd.
- disable normalization with "normalize 0" or "normalize 1"
Yes, disabling this could be a good idea! You could also add a "gain 0-1" message for manual control.
Its worth noting that impulse responses are usually whack without gain normalization.... like factors of hundreds to millions off a usable signal.
Features I'm considering (let me know if they sound useful):
- load from an array instead of from disk (no gain normalization?)
Very good.
- It wouldn't be hard to enable MxN convolution if that floats
somebody's boat.
I am sure if you come up with a convolution as efficient and flexible as jconv by Fons within Pd, then soon a multichannel use and hence request will come up fast.
I'd be interested in what flexibility means in this context, it might give me some good ideas for features to add. Efficiency-wise, last time I benchmarked its more efficient than jconv, but the difference is offset by less graceful degradation under CPU load (I convolve in background threads to preserve realtime in the main thread while avoiding an irritating patent that's going to expire soon...).
WRT to Pd's audio scheduling... are Pd signal externals held to realtime or can my dsp call vary the number of cycles it takes by 100% from call to call? VST seems to do ok with this, but AudioUnits get scheduled to run at the very last instant they possibly could. If Pd can have some variance, I can drop the threads and improve the external's degradation under high CPU load.
thanks for the feedback (also, is the best list for this kind of feedback?),
-Seth
Also, since convolution is much more efficient around block sizes of 256 or 512, perhaps I should default to one of these, buffer a little, and have a "runatpdblocksize" message or somesuch?
I still have not understood if/how the user can set the duration of the first partition of you partitioned convolution, and how these partitions are structured in their (possibly increasing) sizes. Since this first paramter will define the latency-vs-CPU tradeoff it should not be preset by the developers.
I guess this is what I was asking. I support a few "block pattern" partitioning schemes (they're pluggable, its very easy to add a new one), I could export the choice of these to the end-user, including the option of what block size to start with - the minimuum block size of course being Pd's current block size.
My guess is, in the wild, most "pd users" are using Pd-extended, and ships with a 20msec default delay (dunno if this is inherited from vanilla, or overridden by the distro, but either way, same effect: most pd installs probably run at 20msec).
I'm all for allowing configuration of these important parameters, but I want the external to do something sane out of the box. My guess is 64 sample blocks (~20msec) is more abusive CPU-wise than most people expect out-of-the-box, so I'm probably going to default to a partitioning that looks like:
256, 512, 1024, 2048, 4096, 4096, ..., 4096
And allow people to set a different partitioning scheme, including reducing the initial partition size, if they want. That make good sense?
-Seth
P.
PS: Pd and Pd-extended use the same core, audio engine. You might want to consider Pd-extended as vanilla Pd with a folder full of precompiled externals.
On Mon, Apr 4, 2011 at 7:48 PM, Seth Nickell seth@meatscience.net wrote:
- Anyone have requests for features/api? Its currently simplistic:
- takes a "read FILENAME" message, loads the file, does a test convolution against pink noise to normalize the gain to something sane
Is this done within the main Pd audio thread?
The convolution engine has support for doing it either on the calling thread, or a background thread. I'm thinking of default to a background thread. That seem like the right move?
- caches the last N impulse responses, as the test convolution takes a little time - allows setting the cache size with a "cachesize N" message
To make sure I understood this: cachesize is not the size of the first partition of the partitioned convolution, but the cache that tries to avoid audio dropouts when performing the test convolution?
The convolution engine can swap-in a pre-loaded ('cached') IR in realtime without glitching... but it means keeping 2x the Impulse Response data in RAM. To keep the default API simple but useful, I'm defaulting to caching only the last 5 impulse responses in RAM. "cachesize N" lets you increase that number.... lets say in a performance you wanted to use 30 different impulse responses and you have 2GB of ram... should be nbd.
- disable normalization with "normalize 0" or "normalize 1"
Yes, disabling this could be a good idea! You could also add a "gain 0-1" message for manual control.
Its worth noting that impulse responses are usually whack without gain normalization.... like factors of hundreds to millions off a usable signal.
Features I'm considering (let me know if they sound useful): - load from an array instead of from disk (no gain normalization?)
Very good.
- It wouldn't be hard to enable MxN convolution if that floats somebody's boat.
I am sure if you come up with a convolution as efficient and flexible as jconv by Fons within Pd, then soon a multichannel use and hence request will come up fast.
I'd be interested in what flexibility means in this context, it might give me some good ideas for features to add. Efficiency-wise, last time I benchmarked its more efficient than jconv, but the difference is offset by less graceful degradation under CPU load (I convolve in background threads to preserve realtime in the main thread while avoiding an irritating patent that's going to expire soon...).
WRT to Pd's audio scheduling... are Pd signal externals held to realtime or can my dsp call vary the number of cycles it takes by 100% from call to call? VST seems to do ok with this, but AudioUnits get scheduled to run at the very last instant they possibly could. If Pd can have some variance, I can drop the threads and improve the external's degradation under high CPU load.
thanks for the feedback (also, is the best list for this kind of feedback?),
-Seth
On Mon, 4 Apr 2011, Seth Nickell wrote:
Are the DSP calls liable to vary t_signal->s_n (block size) without notification? 64 samples, apparently the default on pd-extended, is doable without buffering for partitioned convolution on a modern computer, but it exacts a pretty high CPU toll, and if I have to handle random blocksize changes, it gets more expensive.
Also, since convolution is much more efficient around block sizes of 256 or 512, perhaps I should default to one of these, buffer a little, and have a "runatpdblocksize" message or somesuch?
There's always a notification. Any change of s_n will result in a new call to the dsp-function.
Note that it's best to make sure that the dsp-function is fairly fast most of the times, because any patching may retrigger the dsp-function in order to recompile the graph.
dsp objects working with some kind of blocks don't have to be using s_n as a setting. I mean that you can accumulate several dsp-blocks in order to make your own kind of bigger block. This is what [fiddle~] and [env~] do, for example.
But some other object classes use s_n as a setting. For example, [fft~] does. I don't know why this is not consistent across all of pd. (I'm not saying either approach is better than the other.)
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
Hi Mathieu,
Thanks, I assumed (without checking :-P) that the dsp call happened every time, didn't realize it was a setup/patching call that registers my "_perform" function with a call graph. Exactly what I need.
I think the difference in approach comes from the needs of the external. fiddle~ probably needs much larger blocks than typical to discriminate between low frequencies. In my case, I can run at 64 sample sizes, but I'll take your whole CPU to do it. It might be smart to default to some internal buffering (say 512), and let people order the external to do really really low latency if they need it and are willing to pay in CPU.
That said, Peter reminded me of an optimization that I hadn't implemented yet. AudioUnits are rarely asked to run below 128 sample block sizes, so it didn't make sense for the AU, and I forgot that it was on the TODO list from 2 years ago. ;-) By convolving very small blocks in the time domain, and switching to frequency domain for larger blocks, I think we can get excellent CPU usage at very small block sizes too.
-Seth
On Tue, Apr 5, 2011 at 8:49 AM, Mathieu Bouchard matju@artengine.ca wrote:
On Mon, 4 Apr 2011, Seth Nickell wrote:
Are the DSP calls liable to vary t_signal->s_n (block size) without notification? 64 samples, apparently the default on pd-extended, is doable without buffering for partitioned convolution on a modern computer, but it exacts a pretty high CPU toll, and if I have to handle random blocksize changes, it gets more expensive.
Also, since convolution is much more efficient around block sizes of 256 or 512, perhaps I should default to one of these, buffer a little, and have a "runatpdblocksize" message or somesuch?
There's always a notification. Any change of s_n will result in a new call to the dsp-function.
Note that it's best to make sure that the dsp-function is fairly fast most of the times, because any patching may retrigger the dsp-function in order to recompile the graph.
dsp objects working with some kind of blocks don't have to be using s_n as a setting. I mean that you can accumulate several dsp-blocks in order to make your own kind of bigger block. This is what [fiddle~] and [env~] do, for example.
But some other object classes use s_n as a setting. For example, [fft~] does. I don't know why this is not consistent across all of pd. (I'm not saying either approach is better than the other.)
_______________________________________________________________________ | Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
On Tue, Apr 5, 2011 at 2:33 PM, Seth Nickell seth@meatscience.net wrote:
Hi Mathieu,
Thanks, I assumed (without checking :-P) that the dsp call happened every time, didn't realize it was a setup/patching call that registers my "_perform" function with a call graph. Exactly what I need.
I think the difference in approach comes from the needs of the external. fiddle~ probably needs much larger blocks than typical to discriminate between low frequencies. In my case, I can run at 64 sample sizes, but I'll take your whole CPU to do it. It might be smart to default to some internal buffering (say 512), and let people order the external to do really really low latency if they need it and are willing to pay in CPU.
Here's where your users' choice of block sizes comes in--if your user puts a partitioned convolution external into a canvas with block size 64, it means to be low-latency. If the user puts it in with [block~ 1024], then the buffering is defined.
Pd means to be ~user~programmable and modular. The more you try to monolith your externals, the worse they work (I've done this). I know I'm not expressing it well, but I hope the point comes through.
That said, Peter reminded me of an optimization that I hadn't implemented yet. AudioUnits are rarely asked to run below 128 sample block sizes, so it didn't make sense for the AU, and I forgot that it was on the TODO list from 2 years ago. ;-) By convolving very small blocks in the time domain, and switching to frequency domain for larger blocks, I think we can get excellent CPU usage at very small block sizes too.
It sounds like you'd have a bit of a problem without first profiling the system or having known profiles for different hardware. Can you tell me more about your partitioning method (just the math)?
-Seth
On Tue, Apr 5, 2011 at 8:49 AM, Mathieu Bouchard matju@artengine.ca wrote:
On Mon, 4 Apr 2011, Seth Nickell wrote:
Are the DSP calls liable to vary t_signal->s_n (block size) without notification? 64 samples, apparently the default on pd-extended, is doable without buffering for partitioned convolution on a modern computer, but it exacts a pretty high CPU toll, and if I have to handle random blocksize changes, it gets more expensive.
Also, since convolution is much more efficient around block sizes of 256 or 512, perhaps I should default to one of these, buffer a little, and
have
a "runatpdblocksize" message or somesuch?
There's always a notification. Any change of s_n will result in a new
call
to the dsp-function.
Note that it's best to make sure that the dsp-function is fairly fast
most
of the times, because any patching may retrigger the dsp-function in
order
to recompile the graph.
dsp objects working with some kind of blocks don't have to be using s_n
as a
setting. I mean that you can accumulate several dsp-blocks in order to
make
your own kind of bigger block. This is what [fiddle~] and [env~] do, for example.
But some other object classes use s_n as a setting. For example, [fft~] does. I don't know why this is not consistent across all of pd. (I'm not saying either approach is better than the other.)
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Hi Charles,
I have a few partitioning methods. I used to do profiling when you first load the plugin, to determine the optimal partitioning, but found that on intel/amd cpus with sse3, it didn't vary much, and just hardcoded a simple rule set for when to use each partitioning style. In the more cross-platform context of pd, I think that profiling code might make sense again, I'll see if I can resurrect it.
-Seth
On Tue, Apr 5, 2011 at 5:04 PM, Charles Henry czhenry@gmail.com wrote:
On Tue, Apr 5, 2011 at 2:33 PM, Seth Nickell seth@meatscience.net wrote:
Hi Mathieu,
Thanks, I assumed (without checking :-P) that the dsp call happened every time, didn't realize it was a setup/patching call that registers my "_perform" function with a call graph. Exactly what I need.
I think the difference in approach comes from the needs of the external. fiddle~ probably needs much larger blocks than typical to discriminate between low frequencies. In my case, I can run at 64 sample sizes, but I'll take your whole CPU to do it. It might be smart to default to some internal buffering (say 512), and let people order the external to do really really low latency if they need it and are willing to pay in CPU.
Here's where your users' choice of block sizes comes in--if your user puts a partitioned convolution external into a canvas with block size 64, it means to be low-latency. If the user puts it in with [block~ 1024], then the buffering is defined.
Pd means to be ~user~programmable and modular. The more you try to monolith your externals, the worse they work (I've done this). I know I'm not expressing it well, but I hope the point comes through.
That said, Peter reminded me of an optimization that I hadn't implemented yet. AudioUnits are rarely asked to run below 128 sample block sizes, so it didn't make sense for the AU, and I forgot that it was on the TODO list from 2 years ago. ;-) By convolving very small blocks in the time domain, and switching to frequency domain for larger blocks, I think we can get excellent CPU usage at very small block sizes too.
It sounds like you'd have a bit of a problem without first profiling the system or having known profiles for different hardware. Can you tell me more about your partitioning method (just the math)?
-Seth
On Tue, Apr 5, 2011 at 8:49 AM, Mathieu Bouchard matju@artengine.ca wrote:
On Mon, 4 Apr 2011, Seth Nickell wrote:
Are the DSP calls liable to vary t_signal->s_n (block size) without notification? 64 samples, apparently the default on pd-extended, is doable without buffering for partitioned convolution on a modern computer, but it exacts a pretty high CPU toll, and if I have to handle random blocksize changes, it gets more expensive.
Also, since convolution is much more efficient around block sizes of 256 or 512, perhaps I should default to one of these, buffer a little, and have a "runatpdblocksize" message or somesuch?
There's always a notification. Any change of s_n will result in a new call to the dsp-function.
Note that it's best to make sure that the dsp-function is fairly fast most of the times, because any patching may retrigger the dsp-function in order to recompile the graph.
dsp objects working with some kind of blocks don't have to be using s_n as a setting. I mean that you can accumulate several dsp-blocks in order to make your own kind of bigger block. This is what [fiddle~] and [env~] do, for example.
But some other object classes use s_n as a setting. For example, [fft~] does. I don't know why this is not consistent across all of pd. (I'm not saying either approach is better than the other.)
_______________________________________________________________________ | Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
On Apr 4, 2011, at 10:48 PM, Seth Nickell wrote:
- Anyone have requests for features/api? Its currently simplistic:
- takes a "read FILENAME" message, loads the file, does a test
convolution against pink noise to normalize the gain to something
saneIs this done within the main Pd audio thread?
The convolution engine has support for doing it either on the calling thread, or a background thread. I'm thinking of default to a background thread. That seem like the right move?
Pd has its own scheduling system which is best to stick to as long as
you can so that you can keep the deterministic operation intact. For
convolution, I can't see a reason to use a thread. It adds complexity
and more code to run, but if the CPU is overtaxed by realtime
convolution processing, you are going to get an interruption in the
audio regardless of whether the processing is in a thread or not.
.hc
- caches the last N impulse responses, as the test convolution
takes a little time
- allows setting the cache size with a "cachesize N" message
To make sure I understood this: cachesize is not the size of the
first partition of the partitioned convolution, but the cache that tries
to avoid audio dropouts when performing the test convolution?The convolution engine can swap-in a pre-loaded ('cached') IR in realtime without glitching... but it means keeping 2x the Impulse Response data in RAM. To keep the default API simple but useful, I'm defaulting to caching only the last 5 impulse responses in RAM. "cachesize N" lets you increase that number.... lets say in a performance you wanted to use 30 different impulse responses and you have 2GB of ram... should be nbd.
- disable normalization with "normalize 0" or "normalize 1"
Yes, disabling this could be a good idea! You could also add a
"gain 0-1" message for manual control.Its worth noting that impulse responses are usually whack without gain normalization.... like factors of hundreds to millions off a usable signal.
Features I'm considering (let me know if they sound useful):
- load from an array instead of from disk (no gain
normalization?)
Very good.
- It wouldn't be hard to enable MxN convolution if that floats
somebody's boat.
I am sure if you come up with a convolution as efficient and
flexible as jconv by Fons within Pd, then soon a multichannel use and hence
request will come up fast.I'd be interested in what flexibility means in this context, it might give me some good ideas for features to add. Efficiency-wise, last time I benchmarked its more efficient than jconv, but the difference is offset by less graceful degradation under CPU load (I convolve in background threads to preserve realtime in the main thread while avoiding an irritating patent that's going to expire soon...).
WRT to Pd's audio scheduling... are Pd signal externals held to realtime or can my dsp call vary the number of cycles it takes by 100% from call to call? VST seems to do ok with this, but AudioUnits get scheduled to run at the very last instant they possibly could. If Pd can have some variance, I can drop the threads and improve the external's degradation under high CPU load.
thanks for the feedback (also, is the best list for this kind of
feedback?),-Seth
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
As we enjoy great advantages from inventions of others, we should be
glad of an opportunity to serve others by any invention of ours; and
this we should do freely and generously. - Benjamin Franklin
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-04-06 20:26, Hans-Christoph Steiner wrote:
Pd has its own scheduling system which is best to stick to as long as you can so that you can keep the deterministic operation intact. For convolution, I can't see a reason to use a thread. It adds complexity and more code to run, but if the CPU is overtaxed by realtime convolution processing, you are going to get an interruption in the audio regardless of whether the processing is in a thread or not.
partioned convolutions can gain massively from parallelisation. given that we have more and more CPUs available, i think it is a good thing to try and do a multicore convolution.
otoh, if there is only a single thread doing the convolution, then there is no parallelisation, and thus the only thing gained is complexity.
using "threats" does not mean that things have to get non-deterministic, and indeed a lot of software uses threads and stays completely deterministic.
gjasdr IOhannes
On Apr 6, 2011, at 2:52 PM, IOhannes m zmoelnig wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-04-06 20:26, Hans-Christoph Steiner wrote:
Pd has its own scheduling system which is best to stick to as long as you can so that you can keep the deterministic operation intact. For convolution, I can't see a reason to use a thread. It adds
complexity and more code to run, but if the CPU is overtaxed by realtime convolution processing, you are going to get an interruption in the audio regardless of whether the processing is in a thread or not.partioned convolutions can gain massively from parallelisation. given that we have more and more CPUs available, i think it is a good thing to try and do a multicore convolution.
otoh, if there is only a single thread doing the convolution, then
there is no parallelisation, and thus the only thing gained is complexity.using "threats" does not mean that things have to get non- deterministic, and indeed a lot of software uses threads and stays completely deterministic.
Yes, you can make things deterministic using threads. Coding without
threads, its basically automatically deterministic, but when using
threads, you have to code things right to have it deterministic.
Having multiple threads to support multiple cores definitely makes
sense, so I guess this multi-threaded pd object would just need to
wait for the results of all threads before letting the DSP tick
complete, thereby ensuring deterministic behavior.
.hc
Terrorism is not an enemy. It cannot be defeated. It's a tactic.
It's about as sensible to say we declare war on night attacks and
expect we're going to win that war. We're not going to win the war on
terrorism. - retired U.S. Army general, William Odom
On Wed, 6 Apr 2011, IOhannes m zmoelnig wrote:
using "threats" does not mean that things have to get non-deterministic, and indeed a lot of software uses threads and stays completely deterministic.
Suppose that you launch a second fred on another cpu. How do you synchronise the main fred and the second fred together so that the main fred doesn't have to wait ? Sounds to me like a big issue with multi-fredded applications. You can't guarantee that the second cpu will run the fred when the first fred will want to, because fredding is dependent on the cpu's availability and the OS's scheduler's decisions.
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
On 07/04/11 16:15, Mathieu Bouchard wrote:
Suppose that you launch a second fred on another cpu.
http://www.archive.org/download/GOSUB10-002/GOSUB10-002-02-ClaudiusMaximus--... http://gosub10.org/GOSUB10-002.html
Claude
if you're not realtime, your main process could get yanked by the scheduler too.... seems more like "realtime vs not-realtime" than "threads: yes/no".
On Thu, Apr 7, 2011 at 8:15 AM, Mathieu Bouchard matju@artengine.ca wrote:
On Wed, 6 Apr 2011, IOhannes m zmoelnig wrote:
using "threats" does not mean that things have to get non-deterministic, and indeed a lot of software uses threads and stays completely deterministic.
Suppose that you launch a second fred on another cpu. How do you synchronise the main fred and the second fred together so that the main fred doesn't have to wait ? Sounds to me like a big issue with multi-fredded applications. You can't guarantee that the second cpu will run the fred when the first fred will want to, because fredding is dependent on the cpu's availability and the OS's scheduler's decisions.
_______________________________________________________________________ | Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
--- On Thu, 4/7/11, Mathieu Bouchard matju@artengine.ca wrote:
From: Mathieu Bouchard matju@artengine.ca Subject: Re: [PD] Making a Realtime Convolution External To: "IOhannes m zmoelnig" zmoelnig@iem.at Cc: pd-list@iem.at Date: Thursday, April 7, 2011, 5:15 PM On Wed, 6 Apr 2011, IOhannes m zmoelnig wrote:
using "threats" does not mean that things have to get
non-deterministic,
and indeed a lot of software uses threads and stays
completely
deterministic.
Suppose that you launch a second fred on another cpu. How do you synchronise the main fred and the second fred together so that the main fred doesn't have to wait ? Sounds to me like a big issue with multi-fredded applications. You can't guarantee that the second cpu will run the fred when the first fred will want to, because fredding is dependent on the cpu's availability and the OS's scheduler's decisions.
I think Tim Blechmann addresses this with Supernova: http://lac.linuxaudio.org/2010/recordings/day4_1215_Supernova.ogv (start maybe 5 min. in...)
-Jonathan
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
-----Inline Attachment Follows-----
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
In response to my comment about acoustic mirror sounding muddy.
I think that most of the directx effects sounded muddy at that time.
Hi Hans,
The thread in question here would be invoked when a "set" message is sent to the object. In this case, I need to load the Impulse Response from the disk and optionally do a test convolution and normalize it. I'm assuming (yeah, I should just check ;-) if I block on an inlet, I'm blocking the whole audio thread?
Convolution with an increasing block size naturally involves different work-loads on different sample blocks (some sample blocks finish a large block that can then be processed, some don't). If the scheduler is picky/precise enough (AudioUnits is), you can't get away with this "ragged work load". Its not a matter of decreasing CPU usage - of course scheduling things to run in a worker thread increases CPU usage a little - its a matter of keeping the _perform call CPU cycles consistent.
-Seth
On Wed, Apr 6, 2011 at 11:26 AM, Hans-Christoph Steiner hans@at.or.at wrote:
On Apr 4, 2011, at 10:48 PM, Seth Nickell wrote:
- Anyone have requests for features/api? Its currently simplistic:
- takes a "read FILENAME" message, loads the file, does a test convolution against pink noise to normalize the gain to something sane
Is this done within the main Pd audio thread?
The convolution engine has support for doing it either on the calling thread, or a background thread. I'm thinking of default to a background thread. That seem like the right move?
Pd has its own scheduling system which is best to stick to as long as you can so that you can keep the deterministic operation intact. For convolution, I can't see a reason to use a thread. It adds complexity and more code to run, but if the CPU is overtaxed by realtime convolution processing, you are going to get an interruption in the audio regardless of whether the processing is in a thread or not.
.hc
- caches the last N impulse responses, as the test convolution takes a little time - allows setting the cache size with a "cachesize N" message
To make sure I understood this: cachesize is not the size of the first partition of the partitioned convolution, but the cache that tries to avoid audio dropouts when performing the test convolution?
The convolution engine can swap-in a pre-loaded ('cached') IR in realtime without glitching... but it means keeping 2x the Impulse Response data in RAM. To keep the default API simple but useful, I'm defaulting to caching only the last 5 impulse responses in RAM. "cachesize N" lets you increase that number.... lets say in a performance you wanted to use 30 different impulse responses and you have 2GB of ram... should be nbd.
- disable normalization with "normalize 0" or "normalize 1"
Yes, disabling this could be a good idea! You could also add a "gain 0-1" message for manual control.
Its worth noting that impulse responses are usually whack without gain normalization.... like factors of hundreds to millions off a usable signal.
Features I'm considering (let me know if they sound useful): - load from an array instead of from disk (no gain normalization?)
Very good.
- It wouldn't be hard to enable MxN convolution if that floats somebody's boat.
I am sure if you come up with a convolution as efficient and flexible as jconv by Fons within Pd, then soon a multichannel use and hence request will come up fast.
I'd be interested in what flexibility means in this context, it might give me some good ideas for features to add. Efficiency-wise, last time I benchmarked its more efficient than jconv, but the difference is offset by less graceful degradation under CPU load (I convolve in background threads to preserve realtime in the main thread while avoiding an irritating patent that's going to expire soon...).
WRT to Pd's audio scheduling... are Pd signal externals held to realtime or can my dsp call vary the number of cycles it takes by 100% from call to call? VST seems to do ok with this, but AudioUnits get scheduled to run at the very last instant they possibly could. If Pd can have some variance, I can drop the threads and improve the external's degradation under high CPU load.
thanks for the feedback (also, is the best list for this kind of feedback?),
-Seth
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
As we enjoy great advantages from inventions of others, we should be glad of an opportunity to serve others by any invention of ours; and this we should do freely and generously. - Benjamin Franklin
On Mon, 4 Apr 2011, Peter Plessas wrote:
This would be of interest for all Pd users, no matter if they like their externals included in a distribution of Pd ('extended') or manuall adding them to their vanilla Pd.
But pd-extended is not merely a bundling of externals.
For example, the [initbang] internal class is not in vanilla and is not possible as an external.
There are also differences about rendering of boxes and fonts, if you've seen any screenshots.
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
Hi Seth,
On 5 Apr 2011, at 01:54, Seth Nickell wrote:
I'm planning to release our realtime convolution engine (extracted from http://meatscience.net/pages/convolution-reverb) as a GPLed Pd external.
What is the advantage of this over Ben Saylor's [partconv~] external, which provides partitioned convolution?
Jamie
Hi Jamie,
Just scanned the source... big difference would be performance, and if you're picky (you have to be pretty picky, honestly), some difference in accuracy due to floating point's reduced precision at large/small values. Convolution is still expensive enough for performance to really matter.
the biggies:
- partconv implements a single fixed block size, but freq domain
convolution is faster by far on bigger blocks (peak on a core duo is near 4k sample blocks). implementing growing block sizes makes a big difference to low latency performance (e.g. 64 64 128 128 256 256 512 512 1024 1024 2048 2048 4096 4096), as you can get low latency while most of your convolutions operating on the ideal high-performance block size.
- vectorization (sse/altivec) of partconv would give a 2-3.5x performance boost
-seth
On Tue, Apr 5, 2011 at 8:26 AM, Jamie Bullock jamie@postlude.co.uk wrote:
Hi Seth,
On 5 Apr 2011, at 01:54, Seth Nickell wrote:
I'm planning to release our realtime convolution engine (extracted from http://meatscience.net/pages/convolution-reverb) as a GPLed Pd external.
What is the advantage of this over Ben Saylor's [partconv~] external, which provides partitioned convolution?
Jamie
On Mon, 4 Apr 2011, Seth Nickell wrote:
- I'd love to build a granular convolution engine.... takes two
real-time signals, and extracts grains from one to convolve against the other. Anyone have ideas about this?
What's the fundamental difference between this and a windowed FFT convolution engine ?
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
On Tue, Apr 5, 2011 at 8:38 AM, Mathieu Bouchard matju@artengine.ca wrote:
On Mon, 4 Apr 2011, Seth Nickell wrote:
- I'd love to build a granular convolution engine.... takes two real-time
signals, and extracts grains from one to convolve against the other. Anyone have ideas about this?
What's the fundamental difference between this and a windowed FFT convolution engine ?
Big difference would be stochastic grain selection (with inputs/control over the selection tendencies), but it'd definitely start as a straight-up windowed fft convolution engine. E.g. one parameter that could make for interesting selections is to hunt for decaying peaks, and favor using those to get a "crisper output" instead of the haze that results from windowed fft convolution.
-seth
Hi Seth,
In terms of feature requests, since you are doing all the work already, it would be nice to have 1x4 mode, meaning one input ->4 convolutions->4 outputs. That would be great for ambisonic (b-format) 4-channel room impulse responses.
Regards, Archontis
On 4/5/11 3:54 AM, Seth Nickell wrote:
I'm planning to release our realtime convolution engine (extracted from http://meatscience.net/pages/convolution-reverb) as a GPLed Pd external.
It currently accepts 4-channel ('true stereo'), two channel or mono impulse responses, with stereo or mono output. Performance is excellent if you have SSE3 and has a fallback in case you don't, and it aims for accuracy (basically that means multi-stage scaling to keep floats within healthy sizes).
- I'd love to swipe the convolve~ external name, currently installed
by mjlib as part of pd-extended. convolve~ from mjlib appears to be a copy of pin~ ? so I think it could be taken? Maybe I mis-read the code. I've cc'ed mark who can probably clarify.
- Anyone have requests for features/api? Its currently simplistic:
- takes a "read FILENAME" message, loads the file, does a test
convolution against pink noise to normalize the gain to something sane - caches the last N impulse responses, as the test convolution takes a little time - allows setting the cache size with a "cachesize N" message - disable normalization with "normalize 0" or "normalize 1"
Features I'm considering (let me know if they sound useful): - load from an array instead of from disk (no gain normalization?) - It wouldn't be hard to enable MxN convolution if that floats somebody's boat.
I can compile/test on Mac& Linux, anyone up for helping me with Windows?
Would this be of interest for Pd-extended?
I'd love to build a granular convolution engine.... takes two
real-time signals, and extracts grains from one to convolve against the other. Anyone have ideas about this?
thanks all,
-Seth
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
-
Archontis Politis -
Billy Stiltner -
Charles Henry -
Claude Heiland-Allen -
Hans-Christoph Steiner -
IOhannes m zmoelnig -
Jamie Bullock -
Jonathan Wilkes -
Mathieu Bouchard -
Peter Plessas -
Seth Nickell