Howdy Miller,
This looks great and is definitely something we've been excited about.
On May 11, 2014, at 6:00 AM, pd-dev-request@iem.at wrote:
To Pd developers...
I've adapted Pd to nternally use a "pdinstance" structure to allow multiple schedulers (including DSP chains) to run in one address space. With slight modification (see below) libpd can use this to run separate patches asynchronously.
I tried to "instance-ize" all Pd symbols to keep them private but ran into seemingly insoluble problems, so am leaving all symbols global for now. (So patches used in pdlib, if they want to be usable in a multiple-instance scenaro, should protect all send/receive/table/delay/etc named objects with a $[0-9] somewhere.
Judging from quickly scanning the test code, the main difference on our end would be switching context between instances?
I was brainstorming a future api for multiple instances along the lines of simply adding a instance index or pointer version of each of the existing libpd api calls, such as:
int libpd_bang(const char *destination); // current function int libpd_bang(t_pdinstance* instance, const char *destination); // additional instance version (probably better to use a pointer, etc)
We could then call pd_setinstance, pdinstance_new etc internally ...
t_pdinstance* libpd_init(); // default, single use case; sets internal default instance pointer and returns it int libpd_init(t_pdinstance* instance); // creates a new instance, returns 0 on success
... but in the end perhaps simply adding the new functions directly is simpler:
t_pdinstance* libpd_init(); // returns default internal instance or null on failure
t_pdinstance* libpd_new_instance(); void libpd_set_instance(t_pdinstance* instance);
I didn't look hard, but I thnk pdlib now will need a way to send $ arguments to patches via libpd_openfile() somehow. Also, libpd_init_audio() will want to make number of channels per-instance, etc. I've put some relevant comments in the test program I'll include below (and sorry for the length of this mail!)
Ok cool. We've got something to go on, great!
-------- Dan Wilcox @danomatika danomatika.com robotcowboy.com
In thinking about this some more, I'm concerned about thread issues if the symbol tables are still global. Would having two instances running on two separate threads (let's say two separate sound card audio threads, for example) work?
On May 15, 2014, at 2:53 PM, Dan Wilcox danomatika@gmail.com wrote:
Howdy Miller,
This looks great and is definitely something we've been excited about.
On May 11, 2014, at 6:00 AM, pd-dev-request@iem.at wrote:
To Pd developers...
I've adapted Pd to nternally use a "pdinstance" structure to allow multiple schedulers (including DSP chains) to run in one address space. With slight modification (see below) libpd can use this to run separate patches asynchronously.
I tried to "instance-ize" all Pd symbols to keep them private but ran into seemingly insoluble problems, so am leaving all symbols global for now. (So patches used in pdlib, if they want to be usable in a multiple-instance scenaro, should protect all send/receive/table/delay/etc named objects with a $[0-9] somewhere.
Judging from quickly scanning the test code, the main difference on our end would be switching context between instances?
I was brainstorming a future api for multiple instances along the lines of simply adding a instance index or pointer version of each of the existing libpd api calls, such as:
int libpd_bang(const char *destination); // current function int libpd_bang(t_pdinstance* instance, const char *destination); // additional instance version (probably better to use a pointer, etc)We could then call pd_setinstance, pdinstance_new etc internally ...
t_pdinstance* libpd_init(); // default, single use case; sets internal default instance pointer and returns it int libpd_init(t_pdinstance* instance); // creates a new instance, returns 0 on success... but in the end perhaps simply adding the new functions directly is simpler:
t_pdinstance* libpd_init(); // returns default internal instance or null on failure t_pdinstance* libpd_new_instance(); void libpd_set_instance(t_pdinstance* instance);I didn't look hard, but I thnk pdlib now will need a way to send $ arguments to patches via libpd_openfile() somehow. Also, libpd_init_audio() will want to make number of channels per-instance, etc. I've put some relevant comments in the test program I'll include below (and sorry for the length of this mail!)
Ok cool. We've got something to go on, great!
Dan Wilcox @danomatika danomatika.com robotcowboy.com
-------- Dan Wilcox @danomatika danomatika.com robotcowboy.com
Also, I'm just throwing that idea out there as an engineering edge case. Obviously it's probably not that expected etc but should that capability be something we might need in the future?
On May 15, 2014, at 3:33 PM, Dan Wilcox danomatika@gmail.com wrote:
In thinking about this some more, I'm concerned about thread issues if the symbol tables are still global. Would having two instances running on two separate threads (let's say two separate sound card audio threads, for example) work?
On May 15, 2014, at 2:53 PM, Dan Wilcox danomatika@gmail.com wrote:
Howdy Miller,
This looks great and is definitely something we've been excited about.
On May 11, 2014, at 6:00 AM, pd-dev-request@iem.at wrote:
To Pd developers...
I've adapted Pd to nternally use a "pdinstance" structure to allow multiple schedulers (including DSP chains) to run in one address space. With slight modification (see below) libpd can use this to run separate patches asynchronously.
I tried to "instance-ize" all Pd symbols to keep them private but ran into seemingly insoluble problems, so am leaving all symbols global for now. (So patches used in pdlib, if they want to be usable in a multiple-instance scenaro, should protect all send/receive/table/delay/etc named objects with a $[0-9] somewhere.
Judging from quickly scanning the test code, the main difference on our end would be switching context between instances?
I was brainstorming a future api for multiple instances along the lines of simply adding a instance index or pointer version of each of the existing libpd api calls, such as:
int libpd_bang(const char *destination); // current function int libpd_bang(t_pdinstance* instance, const char *destination); // additional instance version (probably better to use a pointer, etc)We could then call pd_setinstance, pdinstance_new etc internally ...
t_pdinstance* libpd_init(); // default, single use case; sets internal default instance pointer and returns it int libpd_init(t_pdinstance* instance); // creates a new instance, returns 0 on success... but in the end perhaps simply adding the new functions directly is simpler:
t_pdinstance* libpd_init(); // returns default internal instance or null on failure t_pdinstance* libpd_new_instance(); void libpd_set_instance(t_pdinstance* instance);I didn't look hard, but I thnk pdlib now will need a way to send $ arguments to patches via libpd_openfile() somehow. Also, libpd_init_audio() will want to make number of channels per-instance, etc. I've put some relevant comments in the test program I'll include below (and sorry for the length of this mail!)
Ok cool. We've got something to go on, great!
Dan Wilcox @danomatika danomatika.com robotcowboy.com
Dan Wilcox @danomatika danomatika.com robotcowboy.com
-------- Dan Wilcox @danomatika danomatika.com robotcowboy.com
I think that would work (if Pd was compiled with the "thread lock" enabled) but the two wouldn't be able to run simultaneously; this would reduce efficiency and might make it harder to get low real-time latencies out of a two-sound-card system.
I'm always nervous about multi-threaded real-time systems on the whole (they can be prone to occasional crashes that are hard to re-create). But if there's demand for it I could go ahead and make static storage in PD per-thread which would theoretically make Pd thread-safe. That would work unless for some reason it happened to crash mysteriously and un-debuggably.
cheers M
On Thu, May 15, 2014 at 04:09:11PM -0400, Dan Wilcox wrote:
Also, I'm just throwing that idea out there as an engineering edge case. Obviously it's probably not that expected etc but should that capability be something we might need in the future?
On May 15, 2014, at 3:33 PM, Dan Wilcox danomatika@gmail.com wrote:
In thinking about this some more, I'm concerned about thread issues if the symbol tables are still global. Would having two instances running on two separate threads (let's say two separate sound card audio threads, for example) work?
On May 15, 2014, at 2:53 PM, Dan Wilcox danomatika@gmail.com wrote:
Howdy Miller,
This looks great and is definitely something we've been excited about.
On May 11, 2014, at 6:00 AM, pd-dev-request@iem.at wrote:
To Pd developers...
I've adapted Pd to nternally use a "pdinstance" structure to allow multiple schedulers (including DSP chains) to run in one address space. With slight modification (see below) libpd can use this to run separate patches asynchronously.
I tried to "instance-ize" all Pd symbols to keep them private but ran into seemingly insoluble problems, so am leaving all symbols global for now. (So patches used in pdlib, if they want to be usable in a multiple-instance scenaro, should protect all send/receive/table/delay/etc named objects with a $[0-9] somewhere.
Judging from quickly scanning the test code, the main difference on our end would be switching context between instances?
I was brainstorming a future api for multiple instances along the lines of simply adding a instance index or pointer version of each of the existing libpd api calls, such as:
int libpd_bang(const char *destination); // current function int libpd_bang(t_pdinstance* instance, const char *destination); // additional instance version (probably better to use a pointer, etc)We could then call pd_setinstance, pdinstance_new etc internally ...
t_pdinstance* libpd_init(); // default, single use case; sets internal default instance pointer and returns it int libpd_init(t_pdinstance* instance); // creates a new instance, returns 0 on success... but in the end perhaps simply adding the new functions directly is simpler:
t_pdinstance* libpd_init(); // returns default internal instance or null on failure t_pdinstance* libpd_new_instance(); void libpd_set_instance(t_pdinstance* instance);I didn't look hard, but I thnk pdlib now will need a way to send $ arguments to patches via libpd_openfile() somehow. Also, libpd_init_audio() will want to make number of channels per-instance, etc. I've put some relevant comments in the test program I'll include below (and sorry for the length of this mail!)
Ok cool. We've got something to go on, great!
Dan Wilcox @danomatika danomatika.com robotcowboy.com
Dan Wilcox @danomatika danomatika.com robotcowboy.com
Dan Wilcox @danomatika danomatika.com robotcowboy.com
On 05/16/2014 07:34 AM, Miller Puckette wrote:
I think that would work (if Pd was compiled with the "thread lock" enabled) but the two wouldn't be able to run simultaneously;
the problem is, that if someone made Pd into a (e.g.) VST-plugin (and judging from the responses to your announcement mail, a lot of people definitely would like that to happen), then you cannot make any assumptions about how the host will deal with multiple instances of that plugin. the host may choose to run all it's plugins in parallel :-(
efficiency and might make it harder to get low real-time latencies out of a two-sound-card system.
but this is something the application devs have to worry about, not necessarily Pd.
I'm always nervous about multi-threaded real-time systems on the whole (they can be prone to occasional crashes that are hard to re-create). But if there's demand for it I could go ahead and make static storage in PD per-thread which would theoretically make Pd thread-safe. That would work unless for some reason it happened to crash mysteriously and un-debuggably.
but wouldn't this totally break all threaded objects? e.g. [readsf~]'s file-access thread would see different thread-local variables. you probably can fix [readsf~], but what about externals? e.g. readanysf & Gem use threads quite a bit, flext has thread support,...
i really think that making Pd re-entrant by bundling all static variables in a single per-instance struct is the way to go.
fgmasrd IOhannes
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 17/05/14 12:23, IOhannes m zmölnig wrote:
i really think that making Pd re-entrant by bundling all static variables in a single per-instance struct is the way to go.
+1 (but I would also really like to hear what Peter Brinkmann thinks about this).
Cheers,
Chris.
- -- http://mccormick.cx/
On Sat, May 17, 2014 at 6:23 PM, IOhannes m zmölnig zmoelnig@iem.at wrote:
On 05/16/2014 07:34 AM, Miller Puckette wrote:
I think that would work (if Pd was compiled with the "thread lock"
enabled)
but the two wouldn't be able to run simultaneously;
the problem is, that if someone made Pd into a (e.g.) VST-plugin (and judging from the responses to your announcement mail, a lot of people definitely would like that to happen), then you cannot make any assumptions about how the host will deal with multiple instances of that plugin. the host may choose to run all it's plugins in parallel :-(
But for now, if you want to make a Pd wrapper VST plugin, you can just put locks around all pd calls. I think this first step is a good one. I'm not convinced it's necessary to do the next step, but let's see how this works first. At least it's nice to be able to send and receive pd messages between instances easily. If you really want to run things in parallel, you can always just run Pd in a new process, which is much safer too.
(replying the Dan's message that was in reply to the digest mail)
On Thu, May 22, 2014 at 10:29 AM, Dan Wilcox danomatika@gmail.com wrote:
The issue is, I think, that it feels like we might have the chance to get real multi-context, separate symbols. It would be really great to discuss this and maybe see it happen as opposed to 5 years from now. If it simply involves just lots of symbol renaming, count me first on the list of volunteers. I'll do the dirty work if required.
It feels like the current work is definitely *almost* there and I'm really happy. I'd just like to know what's required to take it that extra step and let's see if we can do it *now*.
If you really want to run things in parallel, you can always just run Pd in a new process, which is much safer too.
Cool but how do I run "just run Pd in a new process" on iOS? I brought up this further separation question to see if the currently solution works for most of the use cases we'd see with libpd.
I don't think there is a public API for starting multiple processes on iOS, the system takes care of that by firing up each app in its own process. These can communicate, though, but they have their own sandbox. Nor do I think multiple processes on a tablet would be very useful.
I could see a use case for using pd instances as dsp processors (similar to a plug-in architecture) on iOS, which would require the same functionality of independant dsp chain and symbol table.
It sounds like you answered that with a "no": "If you really want to run things in parallel...". Remember, at this point, I believe libpd is being mostly used for iOS apps, with desktop being second (you can just run pd+gui there).
People are using libpd in many different ways outside of mobile devices, from embedding in games to installation work, to live performance with visuals... and these areas are more in need of this instance change.
To me, I think the ultimate use case is to be able to fire up two versions of pd in the same processing chain (take vst's in a DAW for example), load each one with identical patches, and have them controlled separately. This would be a fantastic boost for the ability to extend what we can already do with Pd as an audio processing engine, separated from its native GUI. Locking may be necessary in places, but then that is extremely fast these days.
Of course, this all leads to the pdinstance being able to manage the symbol table along with Miller's recent changes, but what are the difficulties in achieving this? It seems like Miller tried and it was more difficult than what we are imagining.
cheers, Rich
On Thu, May 22, 2014 at 2:59 AM, Kjetil Matheussen <k.s.matheussen@gmail.com
wrote:
On Sat, May 17, 2014 at 6:23 PM, IOhannes m zmölnig zmoelnig@iem.atwrote:
On 05/16/2014 07:34 AM, Miller Puckette wrote:
I think that would work (if Pd was compiled with the "thread lock"
enabled)
but the two wouldn't be able to run simultaneously;
the problem is, that if someone made Pd into a (e.g.) VST-plugin (and judging from the responses to your announcement mail, a lot of people definitely would like that to happen), then you cannot make any assumptions about how the host will deal with multiple instances of that plugin. the host may choose to run all it's plugins in parallel :-(
But for now, if you want to make a Pd wrapper VST plugin, you can just put locks around all pd calls. I think this first step is a good one. I'm not convinced it's necessary to do the next step, but let's see how this works first. At least it's nice to be able to send and receive pd messages between instances easily. If you really want to run things in parallel, you can always just run Pd in a new process, which is much safer too.
Pd-dev mailing list Pd-dev@lists.iem.at http://lists.puredata.info/listinfo/pd-dev
-
Chris McCormick -
Dan Wilcox -
IOhannes m zmölnig -
Kjetil Matheussen -
Miller Puckette -
Rich E