Hi Pd dev -
I'm preparing to rework the DSP network to give tilde objects more control over their inputs and outputs, for instance allowing for multi-channel signals and to allow objects to decide for themselves whether to promote float inputs to signals (so that you don't have to say "+~ 0 to get the faster version, and so that I can make the hip/lop/bp/vcf frequency and Q inputs available as signals or as floats).
Of course I mean to make this compatible with existin DSP objects, although for simplicity I'm going to propose one slightly risky move, changing the size of the t_signal structure -- as iohannes mentioned a few years ago, this seems very unlikely to break anyone's tilde objects. The new structure would now look as follows:
typedef struct _signal { int s_n; /* *TOTAL* number of points in the array */ t_sample *s_vec; /* the array */ t_float s_sr; /* *TOTAL* samples per second */ int s_refcount; /* number of times used */ int s_isborrowed; /* whether we're going to borrow our array */ struct _signal *s_borrowedfrom; /* signal to borrow it from */ struct _signal *s_nextfree; /* next in freelist */ struct _signal *s_nextused; /* next in used list */ int s_vecsize; /* allocated size of array in points */ /* *** NEW STUFF *** */ t_float s_rate; /* sample rate */ int s_length; /* number of points in each channel */ int s_nchans; /* number of channels */ int s_overlap; /* number of times each sample will appear */ }
(one question about this... I _could_ take a sightly bigger risk and put the last 3 fields ahead of s_refcount, etc, which I don't think anyone should be using... this would make things look cleaner).
For example, the FFT object's outputs should really have a sample rate of 1/N times the input sample rate, a vector length of 1, and a channel count of N. For compatibility, I'd take the "s_n" field to just be N, although in the future one could optionally use s_length as N and run as many DFTs as there are channels. (This would be incompatible with current practice in wierd situations in which one ran an fft~ objects into another fft~ objects as input - a real bad idea but perhaps the only way in vanilla to time-reverse a signal block by block, so I bet someone is depending on being able to do that :)
Meanwhile, before the DSP routine is called, all signal inputs are populated with vectors by promoting float inputs to signals, all inputs are guaranteed to have the same s_n field, and all outputs are automatically generated to match all the inputs. I want that to be the default option but to allow the object to access non-matching signals, not-filled-in signals (so that it can schedule scalar versions, as in "+~", and to take care of generating its own output signals (which may thus have different sizes from the input signals).
I could then design a "trunk~" object that combines or splits one-channel signals into multichannel ones, and I could extend +~, etc., to know how to add one-channel signals to multichannel ones. Also, clone~ could (optionally) unpack multichannel signals to distribute among copies.
it might also be useful to have the option to ask for the output signals, if auto-generated, never to reuse the same vector as the input; I guess that can be provided if there's a demand for it.
I'm thinking this is a big enough and dangerous enough change that I should do it on a separate branch first. I've got some travel coming up but hope to start coding soonish.
cheers Miller
Hi Miller,
this sounds great! First-class multi-channel support would be a real game changer.
Actually, after Winfried Ritsch told me about the "pd_snake" project, I came up with a couple of ideas on my own. You can find them here: https://git.iem.at/pd/pdsnake/-/blob/master/docu/discussion.txt. Don't know if this aligns with what you are envisioning, but it might give you some inspiration either way :-)
In particular, I would like to point out https://git.iem.at/pd/pdsnake/-/blob/master/docu/discussion.txt#L33-41. This would allow us to create patches where the channel count can be changed dynamically with a single message!
Also, multi-channel signals would give us a chance to vectorize DSP algorithms that are otherwise hard or impossible to optimize. For example, with modern AVX instructions you can compute 8 oscillators or IIR filters for the price of 1. (With proper manual loop unrolling, just like in the "*_perform8" methods, some compilers are able to vectorize it automatically.)
(one question about this... I _could_ take a sightly bigger risk and put the last 3 fields ahead of s_refcount, etc, which I don't think anyone should be using... this would make things look cleaner).
I think this should be fine.
typedef struct _signal { int s_n; /* *TOTAL* number of points in the array */ t_sample *s_vec; /* the array */ t_float s_sr; /* *TOTAL* samples per second */ int s_refcount; /* number of times used */ int s_isborrowed; /* whether we're going to borrow our array */ struct _signal *s_borrowedfrom; /* signal to borrow it from */ struct _signal *s_nextfree; /* next in freelist */ struct _signal *s_nextused; /* next in used list */ int s_vecsize; /* allocated size of array in points */ /* *** NEW STUFF *** */ t_float s_rate; /* sample rate */ int s_length; /* number of points in each channel */ int s_nchans; /* number of channels */ int s_overlap; /* number of times each sample will appear */ }
Personally, I would keep s_n as the number of samples /per channel/. The total number of samples is simply s_n * s_nchans. Existing externals - that do not know about s_nchans - would effectively operate on the first channel and ignore the rest. Newer multi-channel-aware externals, on the other hand, may use all the channels.
I also think that DSP objects would need a new API method to create multi-channel /outputs/. The general idea is that the /input /channel counts are taken from upstream, but the /output /channel counts are specified by the object and passed downstream. (There might be objects where input and output channel count differs; any kind of merger/splitter/mixer objects comes to my mind.)
I think I have some more ideas/notes in one of my notebooks. I can look them up and see if there's something useful.
Anyway, I am quite excited about this!
Cheers,
Christof
On 01.09.2022 21:58, Miller Puckette via Pd-dev wrote:
Hi Pd dev -
I'm preparing to rework the DSP network to give tilde objects more control over their inputs and outputs, for instance allowing for multi-channel signals and to allow objects to decide for themselves whether to promote float inputs to signals (so that you don't have to say "+~ 0 to get the faster version, and so that I can make the hip/lop/bp/vcf frequency and Q inputs available as signals or as floats).
Of course I mean to make this compatible with existin DSP objects, although for simplicity I'm going to propose one slightly risky move, changing the size of the t_signal structure -- as iohannes mentioned a few years ago, this seems very unlikely to break anyone's tilde objects. The new structure would now look as follows:
typedef struct _signal { int s_n; /* *TOTAL* number of points in the array */ t_sample *s_vec; /* the array */ t_float s_sr; /* *TOTAL* samples per second */ int s_refcount; /* number of times used */ int s_isborrowed; /* whether we're going to borrow our array */ struct _signal *s_borrowedfrom; /* signal to borrow it from */ struct _signal *s_nextfree; /* next in freelist */ struct _signal *s_nextused; /* next in used list */ int s_vecsize; /* allocated size of array in points */ /* *** NEW STUFF *** */ t_float s_rate; /* sample rate */ int s_length; /* number of points in each channel */ int s_nchans; /* number of channels */ int s_overlap; /* number of times each sample will appear */ }
(one question about this... I _could_ take a sightly bigger risk and put the last 3 fields ahead of s_refcount, etc, which I don't think anyone should be using... this would make things look cleaner).
For example, the FFT object's outputs should really have a sample rate of 1/N times the input sample rate, a vector length of 1, and a channel count of N. For compatibility, I'd take the "s_n" field to just be N, although in the future one could optionally use s_length as N and run as many DFTs as there are channels. (This would be incompatible with current practice in wierd situations in which one ran an fft~ objects into another fft~ objects as input - a real bad idea but perhaps the only way in vanilla to time-reverse a signal block by block, so I bet someone is depending on being able to do that :)
Meanwhile, before the DSP routine is called, all signal inputs are populated with vectors by promoting float inputs to signals, all inputs are guaranteed to have the same s_n field, and all outputs are automatically generated to match all the inputs. I want that to be the default option but to allow the object to access non-matching signals, not-filled-in signals (so that it can schedule scalar versions, as in "+~", and to take care of generating its own output signals (which may thus have different sizes from the input signals).
I could then design a "trunk~" object that combines or splits one-channel signals into multichannel ones, and I could extend +~, etc., to know how to add one-channel signals to multichannel ones. Also, clone~ could (optionally) unpack multichannel signals to distribute among copies.
it might also be useful to have the option to ask for the output signals, if auto-generated, never to reuse the same vector as the input; I guess that can be provided if there's a demand for it.
I'm thinking this is a big enough and dangerous enough change that I should do it on a separate branch first. I've got some travel coming up but hope to start coding soonish.
cheers Miller
Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev
On 9/2/22 01:00, Christof Ressi wrote:
Hi Miller,
this sounds great! First-class multi-channel support would be a real game changer.
yes. that would be so cool!
typedef struct _signal { int s_n; /* *TOTAL* number of points in the array */ t_sample *s_vec; /* the array */ t_float s_sr; /* *TOTAL* samples per second */
[...]
t_float s_rate; /* sample rate */ int s_length; /* number of points in each channel */ int s_nchans; /* number of channels */ int s_overlap; /* number of times each sample will appear */ }
Personally, I would keep s_n as the number of samples /per channel/. The total number of samples is simply s_n * s_nchans. Existing externals - that do not know about s_nchans - would effectively operate on the first
i think the idea is that with "s_n = s_nchans * s_length" existing externals would automatically process *all* channels.
that's nice if the external does not do any delays or so (as they would automatically become multi-channel aware), but not so nice if they *do* things in the time domain (as there would be weird cross-talk between the channels).
i'm not favouring any of the two approaches, just wanted to point their differences.
i somewhat agree with christof's implication, that it's probably best to not have redundant data in the struct. - 's_n = s_nchans * s_length' (or 's_totalsamples = s_nchans * s_n') - 's_sr = s_rate * s_overlap * s_nchans'
(my issue being, that with redundancy it's more likely to have inconsistent data; what if the struct says 's_n = 128; s_nchans = 3; s_length = 1024'?)
apart from that: probably "s_length" might be called "s_frames" as this seems to be the less ambiguous term.
and i would personally prefer "s_samplerate" and "s_channels". that would make for an easy distinction: the abbreviated names "s_n" and "s_sr" are the convoluted ones, whereas the long names have the data you'd expect.
channel and ignore the rest. Newer multi-channel-aware externals, on the other hand, may use all the channels.
I also think that DSP objects would need a new API method to create multi-channel /outputs/. The general idea is that the /input /channel counts are taken from upstream, but the /output /channel counts are specified by the object and passed downstream. (There might be objects where input and output channel count differs; any kind of merger/splitter/mixer objects comes to my mind.)
+1
vgmasdrf IOhannes
probably "s_length" might be called "s_frames"
I'm not sure about that: in many APIs the word "frame" means one "multi-channel sample", e.g 2 samples for a stereo stream.
Le ven. 2 sept. 2022 à 09:36, IOhannes m zmoelnig zmoelnig@iem.at a écrit :
On 9/2/22 01:00, Christof Ressi wrote:
Hi Miller,
this sounds great! First-class multi-channel support would be a real game changer.
yes. that would be so cool!
typedef struct _signal { int s_n; /* *TOTAL* number of points in the array */ t_sample *s_vec; /* the array */ t_float s_sr; /* *TOTAL* samples per second */
[...]
t_float s_rate; /* sample rate */ int s_length; /* number of points in each channel */ int s_nchans; /* number of channels */ int s_overlap; /* number of times each sample will appear */}
Personally, I would keep s_n as the number of samples /per channel/. The total number of samples is simply s_n * s_nchans. Existing externals - that do not know about s_nchans - would effectively operate on the first
i think the idea is that with "s_n = s_nchans * s_length" existing externals would automatically process *all* channels.
that's nice if the external does not do any delays or so (as they would automatically become multi-channel aware), but not so nice if they *do* things in the time domain (as there would be weird cross-talk between the channels).
i'm not favouring any of the two approaches, just wanted to point their differences.
i somewhat agree with christof's implication, that it's probably best to not have redundant data in the struct.
- 's_n = s_nchans * s_length' (or 's_totalsamples = s_nchans * s_n')
- 's_sr = s_rate * s_overlap * s_nchans'
(my issue being, that with redundancy it's more likely to have inconsistent data; what if the struct says 's_n = 128; s_nchans = 3; s_length = 1024'?)
apart from that: probably "s_length" might be called "s_frames" as this seems to be the less ambiguous term.
and i would personally prefer "s_samplerate" and "s_channels". that would make for an easy distinction: the abbreviated names "s_n" and "s_sr" are the convoluted ones, whereas the long names have the data you'd expect.
channel and ignore the rest. Newer multi-channel-aware externals, on the other hand, may use all the channels.
I also think that DSP objects would need a new API method to create multi-channel /outputs/. The general idea is that the /input /channel counts are taken from upstream, but the /output /channel counts are specified by the object and passed downstream. (There might be objects where input and output channel count differs; any kind of merger/splitter/mixer objects comes to my mind.)
+1
vgmasdrf IOhannes _______________________________________________ Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev
Sorry forget it, "frames" actually corresponds with the comment /* number of points in each channel */".
Le ven. 2 sept. 2022 à 10:55, Antoine Rousseau antoine@metalu.net a écrit :
probably "s_length" might be called "s_frames"
I'm not sure about that: in many APIs the word "frame" means one "multi-channel sample", e.g 2 samples for a stereo stream.
Le ven. 2 sept. 2022 à 09:36, IOhannes m zmoelnig zmoelnig@iem.at a écrit :
On 9/2/22 01:00, Christof Ressi wrote:
Hi Miller,
this sounds great! First-class multi-channel support would be a real game changer.
yes. that would be so cool!
typedef struct _signal { int s_n; /* *TOTAL* number of points in the array */ t_sample *s_vec; /* the array */ t_float s_sr; /* *TOTAL* samples per second */
[...]
t_float s_rate; /* sample rate */ int s_length; /* number of points in each channel */ int s_nchans; /* number of channels */ int s_overlap; /* number of times each sample will appear */}
Personally, I would keep s_n as the number of samples /per channel/.
The
total number of samples is simply s_n * s_nchans. Existing externals - that do not know about s_nchans - would effectively operate on the
first
i think the idea is that with "s_n = s_nchans * s_length" existing externals would automatically process *all* channels.
that's nice if the external does not do any delays or so (as they would automatically become multi-channel aware), but not so nice if they *do* things in the time domain (as there would be weird cross-talk between the channels).
i'm not favouring any of the two approaches, just wanted to point their differences.
i somewhat agree with christof's implication, that it's probably best to not have redundant data in the struct.
- 's_n = s_nchans * s_length' (or 's_totalsamples = s_nchans * s_n')
- 's_sr = s_rate * s_overlap * s_nchans'
(my issue being, that with redundancy it's more likely to have inconsistent data; what if the struct says 's_n = 128; s_nchans = 3; s_length = 1024'?)
apart from that: probably "s_length" might be called "s_frames" as this seems to be the less ambiguous term.
and i would personally prefer "s_samplerate" and "s_channels". that would make for an easy distinction: the abbreviated names "s_n" and "s_sr" are the convoluted ones, whereas the long names have the data you'd expect.
channel and ignore the rest. Newer multi-channel-aware externals, on
the
other hand, may use all the channels.
I also think that DSP objects would need a new API method to create multi-channel /outputs/. The general idea is that the /input /channel counts are taken from upstream, but the /output /channel counts are specified by the object and passed downstream. (There might be objects where input and output channel count differs; any kind of merger/splitter/mixer objects comes to my mind.)
+1
vgmasdrf IOhannes _______________________________________________ Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev
This would be like 'mc' objects in MAX, right? That's awesome!
btw, I see merging season has started, I already updated some documentation files to include the merged changes, please consider merging that soon as well. https://github.com/pure-data/pure-data/pull/1594
cheers
Em sex., 2 de set. de 2022 às 08:03, Antoine Rousseau antoine@metalu.net escreveu:
Sorry forget it, "frames" actually corresponds with the comment /* number of points in each channel */".
Le ven. 2 sept. 2022 à 10:55, Antoine Rousseau antoine@metalu.net a écrit :
probably "s_length" might be called "s_frames"
I'm not sure about that: in many APIs the word "frame" means one "multi-channel sample", e.g 2 samples for a stereo stream.
Le ven. 2 sept. 2022 à 09:36, IOhannes m zmoelnig zmoelnig@iem.at a écrit :
On 9/2/22 01:00, Christof Ressi wrote:
Hi Miller,
this sounds great! First-class multi-channel support would be a real game changer.
yes. that would be so cool!
typedef struct _signal { int s_n; /* *TOTAL* number of points in the array */ t_sample *s_vec; /* the array */ t_float s_sr; /* *TOTAL* samples per second */
[...]
t_float s_rate; /* sample rate */ int s_length; /* number of points in each channel */ int s_nchans; /* number of channels */ int s_overlap; /* number of times each sample will appear */}
Personally, I would keep s_n as the number of samples /per channel/.
The
total number of samples is simply s_n * s_nchans. Existing externals - that do not know about s_nchans - would effectively operate on the
first
i think the idea is that with "s_n = s_nchans * s_length" existing externals would automatically process *all* channels.
that's nice if the external does not do any delays or so (as they would automatically become multi-channel aware), but not so nice if they *do* things in the time domain (as there would be weird cross-talk between the channels).
i'm not favouring any of the two approaches, just wanted to point their differences.
i somewhat agree with christof's implication, that it's probably best to not have redundant data in the struct.
- 's_n = s_nchans * s_length' (or 's_totalsamples = s_nchans * s_n')
- 's_sr = s_rate * s_overlap * s_nchans'
(my issue being, that with redundancy it's more likely to have inconsistent data; what if the struct says 's_n = 128; s_nchans = 3; s_length = 1024'?)
apart from that: probably "s_length" might be called "s_frames" as this seems to be the less ambiguous term.
and i would personally prefer "s_samplerate" and "s_channels". that would make for an easy distinction: the abbreviated names "s_n" and "s_sr" are the convoluted ones, whereas the long names have the data you'd expect.
channel and ignore the rest. Newer multi-channel-aware externals, on
the
other hand, may use all the channels.
I also think that DSP objects would need a new API method to create multi-channel /outputs/. The general idea is that the /input /channel counts are taken from upstream, but the /output /channel counts are specified by the object and passed downstream. (There might be objects where input and output channel count differs; any kind of merger/splitter/mixer objects comes to my mind.)
+1
vgmasdrf IOhannes _______________________________________________ Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev
Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev
Personally, I would keep s_n as the number of samples /per channel/. The total number of samples is simply s_n * s_nchans. Existing externals - that do not know about s_nchans - would effectively operate on the first
i think the idea is that with "s_n = s_nchans * s_length" existing externals would automatically process *all* channels.
that's nice if the external does not do any delays or so (as they would automatically become multi-channel aware), but not so nice if they *do* things in the time domain (as there would be weird cross-talk between the channels).
i'm not favouring any of the two approaches, just wanted to point their differences.
Yup, that's exactly what I had in mind. Any object that is not-multichannel aware and relies on previous input/output samples would produce bogus results. Now, with things like oscillators, the mistake would be very prominent, but with filters it could be very subtle. And subtle errors are the worst of all :-) That's why I think it makes more sense for such objects to only process the first channel.
I think it's worth pointing out that this problem only occurs when the outlet of a multi-channel object connects to the inlet of a legacy object because. The opposite case - legacy object feeding into a multi-channel object - would be perfectly fine.
Now back to the first case: should we consider it a user error? If yes, Pd could even post warnings, but for this we would need a way to tell Pd that an object (class) is multi-channel aware. This could be done with an API method, with a flag for class_new(), or something else...
I'm just brainstorming...
Christof
-
Alexandre Torres Porres -
Antoine Rousseau -
Christof Ressi -
IOhannes m zmoelnig -
Miller Puckette