Hello. My name is Rob Bairos, Director of R+D at www.derivative.ca Ive exchanged a couple brief messages with Peter Brinkmann. Im interested in finding out what would be involved in making pdlib support multiple instances.
From my understanding, the current proposed solution is to take all statics
and globals, encapsulate them in one object, and pass that object to all api calls. Peter further suggested legacy api is maintained by having them call the new api with a default instance object.
I did a little bit of hunting, using objdump on the current dll, to get a rough list of all the globals and statics currently involved.
Im thinking the *_class and *_sym static pointers are in fact constant, and need only one shared instance. That would leave about 320 variables remaining. Many of these variables are constant arrays, strings, etc. And many seem to be used only as a shortcut for passing data between two functions, possibly bringing down the number further.
Im toying with the idea of taking on this task if anyone's interested. I may require some tips and help from the forum, in terms of creating a branch, explanation of some statics etc.
So how feasible is this? Am I on the right track? Thanks very much, Rob Bairos.
PS. Im including the relevant variable list below if anyone's curious.
-------------- rdata Section -------------- iemgui_vu_scale_str iemgui_vu_col iemgui_vu_db2i iemgui_color_hex garray_widgetbehavior glist_valid graph_widgetbehavior drawnumber_widgetbehavior plot_widgetbehavior curve_widgetbehavior text_widgetbehavior s_ s_y s_x s__X s__N s_signal s_anything s_list s_bang s_float s_pointer sys_schedblocksize sys_audioapiopened sys_audioapi sys_flags sys_fontweight sys_font sys_midioutdevlist sys_midiindevlist sys_nmidiin sys_nmidiout sys_hipriority pd_compiledate pd_compiletime pd_version sys_usestdpath sinwrk coswrk init.3151 garray_floattemplatefile garray_arraytemplatefile dollarzero.6960 canvas_newdirectory canvas_newfilename scalar_widgetbehavior gatom_widgetbehavior sys_mutex sched_useaudio audio_advance audio_naudiooutdev audio_naudioindev loaders sys_dllextent2 sys_dllextent sys_fontlist sys_nchout sys_nchin sys_nsoundout sys_nsoundin random_nextseed.60641 p.57111 c1 c0 fpi.5824
-------------- .bss Section -------------- ntab stab lsec gdoo.5287 rsorlist s_errornames faultfontshit agemessage ailingBytesForUTF8 fsetsFromUTF8
-------------- .edata Section -------------- glist_amreloadingabstractions sys_nosleep libpd_midibytehook libpd_polyaftertouchhook libpd_aftertouchhook libpd_pitchbendhook libpd_programchangehook libpd_controlchangehook libpd_noteonhook libpd_messagehook libpd_listhook libpd_floathook libpd_banghook libpd_printhook cos_table bng_widgetbehavior canvas_editing canvas_list canvas_dspstate canvas_whichfind hradio_widgetbehavior hslider_widgetbehavior my_canvas_widgetbehavior my_numbox_widgetbehavior toggle_widgetbehavior vradio_widgetbehavior vslider_widgetbehavior vu_widgetbehavior newest pd_canvasmaker pd_objectmaker glob_pdobject sys_perf sys_idlehook sys_sleepgrain clock_setlist sys_time sys_time_per_dsp_tick sys_advance_samples sys_dacsr sys_outchannels sys_soundin sys_schedadvance sys_inchannels sys_soundout sys_defeatrt sys_externalschedlibname sys_noloadbang sys_debuglevel sys_verbose sys_noautopatch sys_extraflagsstring sys_oldtclversion sys_defaultfont sys_nogui sys_libdir sys_guicmd sys_guisetportnumber sys_externalschedlib sys_extraflags sys_externlist sys_helppath sys_searchpath sys_staticpath sys_printhook sys_printtostderr firstnet rsqrt_mantissatab rsqrt_exptab ugen_currentcontext ugen_sortno signal_usedlist signal_freeborrowed signal_freelist dsp_phase dsp_chainsize dsp_chain array_motion_fatten array_motion_lastx array_motion_yperpix array_motion_xperpix array_motion_initx array_motion_elemsize array_motion_npoints array_motion_template array_motion_wp array_motion_array array_motion_scalar array_motion_glist array_motion_yfield array_motion_xfield array_motion_ycumulative array_motion_xcumulative garray_arraytemplatecanvas warned.7436 gcount.6988 tabcount canvas_newargv canvas_newargc keynamesym.7202 keyupsym.7201 keynumsym.7200 cursorwas.6895 xwas.6894 reenter.6640 canvas_last_glist_y canvas_last_glist_x canvas_last_glist paste_canvas paste_onset canvas_findbuf canvas_find_wholeword canvas_find_index2 canvas_find_index1 canvas_upy canvas_upx canvas_upclicktime canvas_undo_already_set_move canvas_undo_name canvas_undo_canvas canvas_undo_buf canvas_undo_whatnext canvas_undo_fn copy_binbuf rtext_entered warned.6667 drawnumber_motion_firstkey drawnumber_motion_gpointer drawnumber_motion_template drawnumber_motion_wp drawnumber_motion_array drawnumber_motion_scalar drawnumber_motion_glist drawnumber_motion_ycumulative curve_motion_gpointer curve_motion_template curve_motion_wp curve_motion_array curve_motion_scalar curve_motion_glist curve_motion_yper curve_motion_ybase curve_motion_ycumulative curve_motion_xper curve_motion_xbase curve_motion_xcumulative curve_motion_field tryingalready maxclass outlet_eventno stackcount pd_loadingabstraction lastpopped gstack_head idletime.59619 sched_nextmeterpolltime.59581 sched_referencelogicaltime sched_referencerealtime sched_lastoutdb sched_lastindb sched_lastoutclip sched_lastinclip sched_meterson sched_dioredtime sched_diored oss_resync oss_nresync oss_resyncphase sys_histphase sched_didnothing sched_didpoll sched_diddsp sys_histtime sys_histogram sys_quit initted.5834 audio_nextoutchans audio_nextinchans audio_callback_is_open audio_blocksize audio_callback audio_rate audio_audiochoutdev audio_audiooutdev audio_audiochindev audio_audioindev audio_state sys_outmax sys_inmax sys_meters reentered.61489 sys_bytessincelastping sys_waitingforping sys_guibufsize sys_guibuftail sys_guibufhead sys_guibuf sys_guiqueuehead nt_freq nt_inittime sys_socketreceiver inbinbuf sys_guisock sys_maxfd sys_fdpoll sys_nfdpoll sys_loaded sys_choutlist sys_chinlist sys_soundoutdevlist sys_soundindevlist sys_batch sys_listplease sys_main_blocksize sys_main_callback sys_main_advance sys_main_srate sys_version sys_messagelist sys_openlist saidit.5587 errstring errobject error_string error_object gfxstub_binbuf gfxstub_list count.2506 prev.2505 lastwarntime.58421 pleasewarn.58422 argc argm curr argv initialized freelist
Sorry, most of my original post got cut off. Here's the rest (minus the list of symbols) in case its causing a problem:
From my understanding, the current proposed solution is to take all statics
and globals, encapsulate them in one object, and pass that object to all api calls. Peter further suggested legacy api is maintained by having them call the new api with a default instance object.
I did a little bit of hunting, using objdump on the current dll, to get a rough list of all the globals and statics currently involved.
Im thinking the *_class and *_sym static pointers are in fact constant, and need only one shared instance. That would leave about 320 variables remaining. Many of these variables are constant arrays, strings, etc. And many seem to be used only as a shortcut for passing data between two functions, possibly bringing down the number further.
Im toying with the idea of taking on this task if anyone's interested. I may require some tips and help from the forum, in terms of creating a branch, explanation of some statics etc.
So how feasible is this? Am I on the right track? Thanks very much, Rob Bairos.
Excellent plan.
In my branch of libpd on Github, I've solved the Pd multiple instances problem by letting the linker take care of separating the global variables. However, using the linker causing various problems, such as making it very difficult to load externals, and it should probably also be considered a hack. Your plan (the plan I didn't bother doing in my branch) is quite undoubtedly the proper way to do it, and hopefully I would have time to help. At least I would be helping to debug it afterwards, because I would start using this system (in the Radium music editor), instead of my own, to embed Pd instances.
And an advantage to Pd itself might be that the source could be clearer when variables that belongs to the instance, actually are denoted as such in the text.
There is also quite microscopic concern, which is that the added amount of text could make the source more difficult to read, here and there. Maybe a very short variable name for the pd instance, such as "p", "pi', would be a good idea. (i.e. not "pure_data_instance").
On Sat, Dec 7, 2013 at 10:20 PM, Rob Bairos rob@derivative.ca wrote:
Sorry, most of my original post got cut off. Here's the rest (minus the list of symbols) in case its causing a problem:
From my understanding, the current proposed solution is to take all statics and globals, encapsulate them in one object, and pass that object to all api calls. Peter further suggested legacy api is maintained by having them call the new api with a default instance object.
I did a little bit of hunting, using objdump on the current dll, to get a rough list of all the globals and statics currently involved.
Im thinking the *_class and *_sym static pointers are in fact constant, and need only one shared instance. That would leave about 320 variables remaining. Many of these variables are constant arrays, strings, etc. And many seem to be used only as a shortcut for passing data between two functions, possibly bringing down the number further.
Im toying with the idea of taking on this task if anyone's interested. I may require some tips and help from the forum, in terms of creating a branch, explanation of some statics etc.
So how feasible is this? Am I on the right track? Thanks very much, Rob Bairos.
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
On Sun, Dec 8, 2013 at 10:12 AM, Kjetil Matheussen k.s.matheussen@gmail.com wrote:
Excellent plan.
In my branch of libpd on Github, I've solved the Pd multiple instances problem by letting the linker take care of separating the global variables.
A bit unclear. I'm talking about the dynamic linker here, not ld. I.e. Radium dynamically links in libpd for each new instance of pd.
Hi all -
two idea, neither of them as general but perhaps much easier to pull off:
1. make macros like: #define STATIC static __thread
and rely on gcc's per-thread static storage mechanism. This would involve some global search-and-replace action but wouldn't clutter the code too badly. The downside is it would require that each instance of libpd would have to run in its own thread - As Peter pointed out to me, in many situations the programmer can't even determine at compile time whether this would be true or not.
I'm not sure but I think other C compilers besides gcc might support __thread these days.
2. Just make the symbol table and DSP chain per-instance, and leave the rest alone. This only solves a subset of the problem (things like the search path would remain global) but my intuition has it that fixing these two would be enough so that people could practically make patches that don't interfere with each other. (Making the symbol table per-instance would keep things like arrays, send/receives, etc., from cross-talking.)
The result wouldn't be thread-safe; however, combining this with the __thread idea from above would probably work, and then you'd have something that would at least work (although perhaps slightly differently) in same-thread and multi-thread contexts.
These are just ideas - if there's enough interest I can pull (2) off quite easily; (1) would be a global search-and-replace mess that would likely conflict with every source-code patch out there (e.g., all the patches that are applied for Pd extended) so I'd need a real good reason to inflict that one on the world.
cheers Miller
On Sun, Dec 08, 2013 at 10:12:03AM +0100, Kjetil Matheussen wrote:
Excellent plan.
In my branch of libpd on Github, I've solved the Pd multiple instances problem by letting the linker take care of separating the global variables. However, using the linker causing various problems, such as making it very difficult to load externals, and it should probably also be considered a hack. Your plan (the plan I didn't bother doing in my branch) is quite undoubtedly the proper way to do it, and hopefully I would have time to help. At least I would be helping to debug it afterwards, because I would start using this system (in the Radium music editor), instead of my own, to embed Pd instances.
And an advantage to Pd itself might be that the source could be clearer when variables that belongs to the instance, actually are denoted as such in the text.
There is also quite microscopic concern, which is that the added amount of text could make the source more difficult to read, here and there. Maybe a very short variable name for the pd instance, such as "p", "pi', would be a good idea. (i.e. not "pure_data_instance").
On Sat, Dec 7, 2013 at 10:20 PM, Rob Bairos rob@derivative.ca wrote:
Sorry, most of my original post got cut off. Here's the rest (minus the list of symbols) in case its causing a problem:
From my understanding, the current proposed solution is to take all statics and globals, encapsulate them in one object, and pass that object to all api calls. Peter further suggested legacy api is maintained by having them call the new api with a default instance object.
I did a little bit of hunting, using objdump on the current dll, to get a rough list of all the globals and statics currently involved.
Im thinking the *_class and *_sym static pointers are in fact constant, and need only one shared instance. That would leave about 320 variables remaining. Many of these variables are constant arrays, strings, etc. And many seem to be used only as a shortcut for passing data between two functions, possibly bringing down the number further.
Im toying with the idea of taking on this task if anyone's interested. I may require some tips and help from the forum, in terms of creating a branch, explanation of some statics etc.
So how feasible is this? Am I on the right track? Thanks very much, Rob Bairos.
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Hi Miller,
Idea #1 sounds quite good, except that it sounds hacky and that performance might go down notifiable because of thread switching. The extra amount of code necessary to switch threads doesn't sound like too much work.
So I like idea #2 much better. The limitation of only one DSP chain was the only good reason for implementing multiple pd instances for Radium. If you implement #2, that's probably good enough for Radium, and most likely good enough for most others too. At least, it's a very good start.
On Sun, Dec 8, 2013 at 6:53 PM, Miller Puckette msp@ucsd.edu wrote:
Hi all -
two idea, neither of them as general but perhaps much easier to pull off:
- make macros like:
#define STATIC static __thread
and rely on gcc's per-thread static storage mechanism. This would involve some global search-and-replace action but wouldn't clutter the code too badly. The downside is it would require that each instance of libpd would have to run in its own thread - As Peter pointed out to me, in many situations the programmer can't even determine at compile time whether this would be true or not.
I'm not sure but I think other C compilers besides gcc might support __thread these days.
- Just make the symbol table and DSP chain per-instance, and leave the rest
alone. This only solves a subset of the problem (things like the search path would remain global) but my intuition has it that fixing these two would be enough so that people could practically make patches that don't interfere with each other. (Making the symbol table per-instance would keep things like arrays, send/receives, etc., from cross-talking.)
The result wouldn't be thread-safe; however, combining this with the __thread idea from above would probably work, and then you'd have something that would at least work (although perhaps slightly differently) in same-thread and multi-thread contexts.
These are just ideas - if there's enough interest I can pull (2) off quite easily; (1) would be a global search-and-replace mess that would likely conflict with every source-code patch out there (e.g., all the patches that are applied for Pd extended) so I'd need a real good reason to inflict that one on the world.
cheers Miller
On Sun, Dec 08, 2013 at 10:12:03AM +0100, Kjetil Matheussen wrote:
Excellent plan.
In my branch of libpd on Github, I've solved the Pd multiple instances problem by letting the linker take care of separating the global variables. However, using the linker causing various problems, such as making it very difficult to load externals, and it should probably also be considered a hack. Your plan (the plan I didn't bother doing in my branch) is quite undoubtedly the proper way to do it, and hopefully I would have time to help. At least I would be helping to debug it afterwards, because I would start using this system (in the Radium music editor), instead of my own, to embed Pd instances.
And an advantage to Pd itself might be that the source could be clearer when variables that belongs to the instance, actually are denoted as such in the text.
There is also quite microscopic concern, which is that the added amount of text could make the source more difficult to read, here and there. Maybe a very short variable name for the pd instance, such as "p", "pi', would be a good idea. (i.e. not "pure_data_instance").
On Sat, Dec 7, 2013 at 10:20 PM, Rob Bairos rob@derivative.ca wrote:
Sorry, most of my original post got cut off. Here's the rest (minus the list of symbols) in case its causing a problem:
From my understanding, the current proposed solution is to take all statics and globals, encapsulate them in one object, and pass that object to all api calls. Peter further suggested legacy api is maintained by having them call the new api with a default instance object.
I did a little bit of hunting, using objdump on the current dll, to get a rough list of all the globals and statics currently involved.
Im thinking the *_class and *_sym static pointers are in fact constant, and need only one shared instance. That would leave about 320 variables remaining. Many of these variables are constant arrays, strings, etc. And many seem to be used only as a shortcut for passing data between two functions, possibly bringing down the number further.
Im toying with the idea of taking on this task if anyone's interested. I may require some tips and help from the forum, in terms of creating a branch, explanation of some statics etc.
So how feasible is this? Am I on the right track? Thanks very much, Rob Bairos.
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Sorry for the delay. #1 I agree. Seems like a workaround. Ultimately it should be clear in the source code when a static is meant to be shared across threads or not. Seems like something that could be properly implemented in the future?
#2 Sounds good, but Im not familiar with which variables are symbol table related, DSP chain related, etc. Im of the mind, Id just like to put them *all* into one structure, to get a stable release, and then individual variables can be pulled back out in future as the need arises. Id be inclined to throw everything into one structure, and name things according to which file they originated in:
example, firstnet in d_fftroutine.c would live as an entry
struct PDinstance { ... FFT_NET *d_fftroutine_firstnet; ... }
This would allow one to at least see where the variable is used.
#3 Peter mentions that in order to support legacy code, all API calls would need to be mirrored, with and without the pd-instance variable. I don't think C allows for overloading, so would this require a separate name for all the functions? Would supporting two parallel APIs be wanted though, or just lead to confusion? Is this in order to support previously compiled objects (Dlls)?
-Rob
On Mon, Dec 9, 2013 at 5:30 AM, Kjetil Matheussen k.s.matheussen@gmail.comwrote:
Hi Miller,
Idea #1 sounds quite good, except that it sounds hacky and that performance might go down notifiable because of thread switching. The extra amount of code necessary to switch threads doesn't sound like too much work.
So I like idea #2 much better. The limitation of only one DSP chain was the only good reason for implementing multiple pd instances for Radium. If you implement #2, that's probably good enough for Radium, and most likely good enough for most others too. At least, it's a very good start.
On Sun, Dec 8, 2013 at 6:53 PM, Miller Puckette msp@ucsd.edu wrote:
Hi all -
two idea, neither of them as general but perhaps much easier to pull off:
- make macros like:
#define STATIC static __thread
and rely on gcc's per-thread static storage mechanism. This would
involve
some global search-and-replace action but wouldn't clutter the code too
badly.
The downside is it would require that each instance of libpd would have
to
run in its own thread - As Peter pointed out to me, in many situations
the
programmer can't even determine at compile time whether this would be
true
or not.
I'm not sure but I think other C compilers besides gcc might support
__thread
these days.
- Just make the symbol table and DSP chain per-instance, and leave
the rest
alone. This only solves a subset of the problem (things like the search
path
would remain global) but my intuition has it that fixing these two would
be
enough so that people could practically make patches that don't interfere with each other. (Making the symbol table per-instance would keep things like arrays, send/receives, etc., from cross-talking.)
The result wouldn't be thread-safe; however, combining this with the __thread idea from above would probably work, and then you'd have
something
that would at least work (although perhaps slightly differently) in same-thread and multi-thread contexts.
These are just ideas - if there's enough interest I can pull (2) off
quite
easily; (1) would be a global search-and-replace mess that would likely conflict with every source-code patch out there (e.g., all the patches
that
are applied for Pd extended) so I'd need a real good reason to inflict
that
one on the world.
cheers Miller
On Sun, Dec 08, 2013 at 10:12:03AM +0100, Kjetil Matheussen wrote:
Excellent plan.
In my branch of libpd on Github, I've solved the Pd multiple instances problem by letting the linker take care of separating the global variables. However, using the linker causing various problems, such as making it very difficult to load externals, and it should probably also be considered a hack. Your plan (the plan I didn't bother doing in my branch) is quite undoubtedly the proper way to do it, and hopefully I would have time to help. At least I would be helping to debug it afterwards, because I would start using this system (in the Radium music editor), instead of my own, to embed Pd instances.
And an advantage to Pd itself might be that the source could be clearer
when
variables that belongs to the instance, actually are denoted as such in the text.
There is also quite microscopic concern, which is that the added amount of text could make the source more difficult to read, here and there. Maybe a very short variable name for the pd instance, such as "p", "pi', would be a good idea. (i.e. not
"pure_data_instance").
On Sat, Dec 7, 2013 at 10:20 PM, Rob Bairos rob@derivative.ca wrote:
Sorry, most of my original post got cut off. Here's the rest (minus the list of symbols) in case its causing a
problem:
From my understanding, the current proposed solution is to take all
statics
and globals, encapsulate them in one object, and pass that object to all api calls. Peter further suggested legacy api is maintained by having them call
the new
api with a default instance object.
I did a little bit of hunting, using objdump on the current dll, to
get a
rough list of all the globals and statics currently involved.
Im thinking the *_class and *_sym static pointers are in fact
constant, and
need only one shared instance. That would leave about 320 variables remaining. Many of these variables are constant arrays, strings, etc. And many seem to be used only as a shortcut for passing data between
two
functions, possibly bringing down the number further.
Im toying with the idea of taking on this task if anyone's interested. I may require some tips and help from the forum, in terms of creating
a
branch, explanation of some statics etc.
So how feasible is this? Am I on the right track? Thanks very much, Rob Bairos.
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Well, I think "#1" (making statics per-thread) is only necessary on the assumtion that you want Pd running simultaneously in multiple threads; otherwise it would accomplish nothing. I'm not sure but I guess the overhead would be the same as the "one-structure" solution (which is essentially how, I think, the per-thread thing would have to work internally anyhow.)
The idea of throwing all statis/externs into a data structure wouldn't extend to externs, which would be ugly since it would mean the API for internal objects would have to be different than for externs (which would have to make their own data structures, per-Pd-instance, and would have to have some way to look those up from the Pd instance. Also, the idea of having a data structure that you have to change to add anthing static to anything at all inside Pd sounds quite heavy to me.
I'm liking the idea of simply "localizing" symbols and the DSP chain more and more as I think about it... it's nice and self-contained and I think it would help things a lot from what I'm hearing.
cheers Miller
On Tue, Dec 10, 2013 at 01:07:08PM -0500, Rob Bairos wrote:
Sorry for the delay. #1 I agree. Seems like a workaround. Ultimately it should be clear in the source code when a static is meant to be shared across threads or not. Seems like something that could be properly implemented in the future?
#2 Sounds good, but Im not familiar with which variables are symbol table related, DSP chain related, etc. Im of the mind, Id just like to put them *all* into one structure, to get a stable release, and then individual variables can be pulled back out in future as the need arises. Id be inclined to throw everything into one structure, and name things according to which file they originated in:
example, firstnet in d_fftroutine.c would live as an entry
struct PDinstance { ... FFT_NET *d_fftroutine_firstnet; ... }
This would allow one to at least see where the variable is used.
#3 Peter mentions that in order to support legacy code, all API calls would need to be mirrored, with and without the pd-instance variable. I don't think C allows for overloading, so would this require a separate name for all the functions? Would supporting two parallel APIs be wanted though, or just lead to confusion? Is this in order to support previously compiled objects (Dlls)?
-Rob
On Mon, Dec 9, 2013 at 5:30 AM, Kjetil Matheussen k.s.matheussen@gmail.comwrote:
Hi Miller,
Idea #1 sounds quite good, except that it sounds hacky and that performance might go down notifiable because of thread switching. The extra amount of code necessary to switch threads doesn't sound like too much work.
So I like idea #2 much better. The limitation of only one DSP chain was the only good reason for implementing multiple pd instances for Radium. If you implement #2, that's probably good enough for Radium, and most likely good enough for most others too. At least, it's a very good start.
On Sun, Dec 8, 2013 at 6:53 PM, Miller Puckette msp@ucsd.edu wrote:
Hi all -
two idea, neither of them as general but perhaps much easier to pull off:
- make macros like:
#define STATIC static __thread
and rely on gcc's per-thread static storage mechanism. This would
involve
some global search-and-replace action but wouldn't clutter the code too
badly.
The downside is it would require that each instance of libpd would have
to
run in its own thread - As Peter pointed out to me, in many situations
the
programmer can't even determine at compile time whether this would be
true
or not.
I'm not sure but I think other C compilers besides gcc might support
__thread
these days.
- Just make the symbol table and DSP chain per-instance, and leave
the rest
alone. This only solves a subset of the problem (things like the search
path
would remain global) but my intuition has it that fixing these two would
be
enough so that people could practically make patches that don't interfere with each other. (Making the symbol table per-instance would keep things like arrays, send/receives, etc., from cross-talking.)
The result wouldn't be thread-safe; however, combining this with the __thread idea from above would probably work, and then you'd have
something
that would at least work (although perhaps slightly differently) in same-thread and multi-thread contexts.
These are just ideas - if there's enough interest I can pull (2) off
quite
easily; (1) would be a global search-and-replace mess that would likely conflict with every source-code patch out there (e.g., all the patches
that
are applied for Pd extended) so I'd need a real good reason to inflict
that
one on the world.
cheers Miller
On Sun, Dec 08, 2013 at 10:12:03AM +0100, Kjetil Matheussen wrote:
Excellent plan.
In my branch of libpd on Github, I've solved the Pd multiple instances problem by letting the linker take care of separating the global variables. However, using the linker causing various problems, such as making it very difficult to load externals, and it should probably also be considered a hack. Your plan (the plan I didn't bother doing in my branch) is quite undoubtedly the proper way to do it, and hopefully I would have time to help. At least I would be helping to debug it afterwards, because I would start using this system (in the Radium music editor), instead of my own, to embed Pd instances.
And an advantage to Pd itself might be that the source could be clearer
when
variables that belongs to the instance, actually are denoted as such in the text.
There is also quite microscopic concern, which is that the added amount of text could make the source more difficult to read, here and there. Maybe a very short variable name for the pd instance, such as "p", "pi', would be a good idea. (i.e. not
"pure_data_instance").
On Sat, Dec 7, 2013 at 10:20 PM, Rob Bairos rob@derivative.ca wrote:
Sorry, most of my original post got cut off. Here's the rest (minus the list of symbols) in case its causing a
problem:
From my understanding, the current proposed solution is to take all
statics
and globals, encapsulate them in one object, and pass that object to all api calls. Peter further suggested legacy api is maintained by having them call
the new
api with a default instance object.
I did a little bit of hunting, using objdump on the current dll, to
get a
rough list of all the globals and statics currently involved.
Im thinking the *_class and *_sym static pointers are in fact
constant, and
need only one shared instance. That would leave about 320 variables remaining. Many of these variables are constant arrays, strings, etc. And many seem to be used only as a shortcut for passing data between
two
functions, possibly bringing down the number further.
Im toying with the idea of taking on this task if anyone's interested. I may require some tips and help from the forum, in terms of creating
a
branch, explanation of some statics etc.
So how feasible is this? Am I on the right track? Thanks very much, Rob Bairos.
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
On Wed, Dec 11, 2013 at 3:55 AM, Miller Puckette msp@ucsd.edu wrote:
I'm liking the idea of simply "localizing" symbols and the DSP chain more and more as I think about it... it's nice and self-contained and I think it would help things a lot from what I'm hearing.
Perhaps I misunderstand what you talk about, but for Radium, I think a shared symbol table could be better. Then you can share arrays and other things between instances. (And patch specific symbol names should be injected with $0 anyway.)
On Wed, Dec 11, 2013 at 8:52 AM, Kjetil Matheussen k.s.matheussen@gmail.com wrote:
On Wed, Dec 11, 2013 at 3:55 AM, Miller Puckette msp@ucsd.edu wrote:
I'm liking the idea of simply "localizing" symbols and the DSP chain more and more as I think about it... it's nice and self-contained and I think it would help things a lot from what I'm hearing.
Perhaps I misunderstand what you talk about, but for Radium, I think a shared symbol table could be better. Then you can share arrays and other things between instances. (And patch specific symbol names should be injected with $0 anyway.)
Oh, but for sending messages between instances, you would have to queue the messages, and then it starts to get complicated...
"Oh, but for sending messages between instances, you would have to queue
the messages, and then it starts to get complicated.." I think sticking to independent instances with no shared arrays/etc would be a great first step.
"The idea of throwing all statis/externs into a data structure wouldn't
extend to externs, which would be ugly since it would mean the API for internal objects would have to be different than for externs (which would have to make their own data structures, per-Pd-instance, and would have to have some way to look those up from the Pd instance."
Sorry, Im pretty new to this code, whats an extern in pdlib?
"Also, the idea of
having a data structure that you have to change to add anthing static to anything at all inside Pd sounds quite heavy to me."
Well if the engine requires a static variable, it should be treated properly, whether thats implementing it as a global instance structure or some other mechanism, different than simply declaring it static.
"I'm liking the idea of simply "localizing" symbols and the DSP chain more
and more as I think about it... it's nice and self-contained and I think it would help things a lot from what I'm hearing."
Well the first pass would definitely involve avoiding the need for a variable to be static in the first place. My brief skimming indicated that fit quite a few of them.
Does it make more sense for the instance structure to be a set of structures?
On Wed, Dec 11, 2013 at 2:56 AM, Kjetil Matheussen <k.s.matheussen@gmail.com
wrote:
On Wed, Dec 11, 2013 at 8:52 AM, Kjetil Matheussen k.s.matheussen@gmail.com wrote:
On Wed, Dec 11, 2013 at 3:55 AM, Miller Puckette msp@ucsd.edu wrote:
I'm liking the idea of simply "localizing" symbols and the DSP chain
more
and more as I think about it... it's nice and self-contained and I
think it
would help things a lot from what I'm hearing.
Perhaps I misunderstand what you talk about, but for Radium, I think a shared symbol table could be better. Then you can share arrays and other things between instances. (And patch specific symbol names should be injected with $0 anyway.)
Oh, but for sending messages between instances, you would have to queue the messages, and then it starts to get complicated...
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 2013-12-11 20:38, Rob Bairos wrote:
Sorry, Im pretty new to this code, whats an extern in pdlib?
the same as in proper Pd: an "external" aka "plugin" that is, a pre-compiled object loaded on-demand at runtime.
Does it make more sense for the instance structure to be a set of structures?
if we go that route, i'd suggest to use a structure of pointers to structures (so the separate structures can be more easily extended without breaking binary incompatibility).
fgmasdr IOhannes
+1 to localizing the symbol table and dsp chain, that would be a huge step forward towards being able to use pd as a 'plugin''.
What other things would be global other than search path? Audio and midi settings?
cheers, Rich
On Thu, Dec 12, 2013 at 3:49 AM, IOhannes m zmoelnig zmoelnig@iem.atwrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 2013-12-11 20:38, Rob Bairos wrote:
Sorry, Im pretty new to this code, whats an extern in pdlib?
the same as in proper Pd: an "external" aka "plugin" that is, a pre-compiled object loaded on-demand at runtime.
Does it make more sense for the instance structure to be a set of structures?
if we go that route, i'd suggest to use a structure of pointers to structures (so the separate structures can be more easily extended without breaking binary incompatibility).
fgmasdr IOhannes -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.15 (GNU/Linux) Comment: Using GnuPG with Icedove - http://www.enigmail.net/
iQIcBAEBCAAGBQJSqXiCAAoJELZQGcR/ejb4LBgP/1wayhKsCdBOX8in1tklqISK gNR8ChKtsJX5Zju2oqBI7BL5VfIg1NdciWO1hAUhM4HLWG4Wd1/8oICFez+Nggu9 zmYa/lxxIk6XDuKtVCCRfCPYu3WTKbujwd1c7GVcnM2RSNInwwRuHn5wXhsi5HuC 60GtX086gPDwMtLNB3D8YjJxhhgnydP4SV4OUfxVW7USOpwsxQIPzPGInX7ukEE5 Tet+LCYKv27ojbZshJ5jYOp+zaK1vl7gx1Tu7NSGZbpo0pk1tQrXWy+KUlrvgRGL zA/YUe280NWhf8JPBh9VMDCSXpRfU2LJkrrXIWStl/dphX0jj88nSjPu4qpuzFtJ fAY13Noatjy357IvTh5i8YxZhX/Ryp5iuem0O/dzJJelD6KQQ97qYNXTTitcSWd8 bq3nyFoswi8KLt1svil/Ukd9/BAF1jNuNxGGKbpnMfkist+S616kvLd4Luo1exvd NLo86trvie3SMsnTH9itGFOBP9DwKCp3FvNwo/50kS8ubZSbXjmwPjnAGSoD9fNA scZElHJxlXISUA/ah4XTv+0RtPNjjyJhvsMjm18fzdjinziyizYw7n0SLlelCy1U sqsbWlXQYKVS4HG2H5qM6QFZgb9ov+B/z5NVkcyn85C5LXIgL6BrTVk/anWQrbeD P4d0RlArbWhCqWDrOLnn =UyCi -----END PGP SIGNATURE-----
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
why would you make audio settings global? in our particular setup each 'plugin / node' etc has an arbitrary number of inputs/outputs/sample rate.
On Thu, Dec 12, 2013 at 12:31 PM, Rich E reakinator@gmail.com wrote:
+1 to localizing the symbol table and dsp chain, that would be a huge step forward towards being able to use pd as a 'plugin''.
What other things would be global other than search path? Audio and midi settings?
cheers, Rich
On Thu, Dec 12, 2013 at 3:49 AM, IOhannes m zmoelnig zmoelnig@iem.atwrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 2013-12-11 20:38, Rob Bairos wrote:
Sorry, Im pretty new to this code, whats an extern in pdlib?
the same as in proper Pd: an "external" aka "plugin" that is, a pre-compiled object loaded on-demand at runtime.
Does it make more sense for the instance structure to be a set of structures?
if we go that route, i'd suggest to use a structure of pointers to structures (so the separate structures can be more easily extended without breaking binary incompatibility).
fgmasdr IOhannes -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.15 (GNU/Linux) Comment: Using GnuPG with Icedove - http://www.enigmail.net/
iQIcBAEBCAAGBQJSqXiCAAoJELZQGcR/ejb4LBgP/1wayhKsCdBOX8in1tklqISK gNR8ChKtsJX5Zju2oqBI7BL5VfIg1NdciWO1hAUhM4HLWG4Wd1/8oICFez+Nggu9 zmYa/lxxIk6XDuKtVCCRfCPYu3WTKbujwd1c7GVcnM2RSNInwwRuHn5wXhsi5HuC 60GtX086gPDwMtLNB3D8YjJxhhgnydP4SV4OUfxVW7USOpwsxQIPzPGInX7ukEE5 Tet+LCYKv27ojbZshJ5jYOp+zaK1vl7gx1Tu7NSGZbpo0pk1tQrXWy+KUlrvgRGL zA/YUe280NWhf8JPBh9VMDCSXpRfU2LJkrrXIWStl/dphX0jj88nSjPu4qpuzFtJ fAY13Noatjy357IvTh5i8YxZhX/Ryp5iuem0O/dzJJelD6KQQ97qYNXTTitcSWd8 bq3nyFoswi8KLt1svil/Ukd9/BAF1jNuNxGGKbpnMfkist+S616kvLd4Luo1exvd NLo86trvie3SMsnTH9itGFOBP9DwKCp3FvNwo/50kS8ubZSbXjmwPjnAGSoD9fNA scZElHJxlXISUA/ah4XTv+0RtPNjjyJhvsMjm18fzdjinziyizYw7n0SLlelCy1U sqsbWlXQYKVS4HG2H5qM6QFZgb9ov+B/z5NVkcyn85C5LXIgL6BrTVk/anWQrbeD P4d0RlArbWhCqWDrOLnn =UyCi -----END PGP SIGNATURE-----
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
On Thu, Dec 12, 2013 at 1:16 PM, Rob Bairos rob@derivative.ca wrote:
why would you make audio settings global? in our particular setup each 'plugin / node' etc has an arbitrary number of inputs/outputs/sample rate.
I wouldn't, but I think they might already be. I was asking what other vars would remain global in Miller's proposed solution, giving audio setttings as an example of something that you'd probably want to also be local. I don't know if there are others.
On Thu, Dec 12, 2013 at 6:31 PM, Rich E reakinator@gmail.com wrote:
+1 to localizing the symbol table and dsp chain, that would be a huge step forward towards being able to use pd as a 'plugin''.
Pd can already be used as a plugin if you use libpds (with an "s"). See https://github.com/kmatheussen/libpd libpds has multiple instances and GUI, and can easily be made into a LADSPA plugin, for instance. Radium uses libpds to plug in Pd.
However, libpds is very resource hungry, difficult to load externals, and currently only works for linux, so localized symbol table and dsp chain instances would be much better.
-
IOhannes m zmoelnig -
Kjetil Matheussen -
Miller Puckette -
Rich E -
Rob Bairos