hi miller et all,
the current implementation of Pd's loading mechanism works as follows:
- each registered loader is asked whether they can handle a given objectclass - each loader in turn searches the entire (canvas-enhanced) path for files matching their pattern (eg.g. "*.pd_linux", or "*.pd_lua") - if all fails, Pd searches the entire (canvas-enhanced) path for abstractions.
e.g. /A/x.pd_linux /B/x.pd_linux /C/x.pd_linux /A/x.pd_lua /B/x.pd_lua /C/x.pd_lua /A/x.pd /B/x.pd /C/x.pd
this has a number of disadvantages, mainly:
- the associated loader has a higher priority then the path. this means that it is nigh¹ impossible to override a binary external (e.g. "*.pd_darwin") installed system-wide (e.g. in extra/) with a user-provided alternative (e.g. a pd_lua implementation in the current patch path).
- abstractions (arguably the most used external classes) have an extra load time penalty, since each time a new (abstraction-provided) object is created, Pd will first search the entire path for binaries, lua-scripts and what not.
iirc, this has been discussed at length, and the proper solution for this is to change the loading mechanism, so that paths have a higher that loaders: Pd shall iterate over each search-path and ask the loaders whether there is something for them in there.
this way an abstraction in a "-path" enhanced searchpath is always found early in the search, AND it can shadow an external in a later search path.
e.g. /A/x.pd_linux /A/x.pd_lua /A/x.pd /B/x.pd_linux /B/x.pd_lua /B/x.pd /C/x.pd_linux /C/x.pd_lua /C/x.pd
so i have taken the liberty and implement that (i'll submit the patches in a second).
here's a few observations:
compatibility with legacy loaders === the new implementation needs an API that tells the loaders where they should look for. rather than create a new API, i used the existing one and enhanced the callback function with a new argument <path>, so we now have: int loader_callback(t_canvas*c, char*name, char*path);
the good thing about this is that it is actually backward compatible (at least with sane calling conventions, as on my tested linux system). legacy loaders will simply ignore the <path> argument, and continue to search their files in the entire path. this will result in an overall performance degredation (while loading objects), as the legacy loaders will call the entire search path for each element in the search path :-( the good news is that: - the legacy loaders will continue to work! - the performance degredation will only happen if legacy loaders are involved. - the performance degredation will be at *load time*, so should not be too important - there are only very few (known) loaders, so they should be fixed in no time
abstraction loading === one big change is that abstractions are now loaded via a proper "loader" (rather than via some special handling as fallback) this implies that whenever the abstraction-loader wins (that is, a given objectname can first be resolved via an abstraction), this objectname is tagged as "abstraction". when another object of the same name is created, the loader competition will not take place any more, and instead the object will be loaded immediately as "abstraction" (the current implementation then still searches the entire path for the .pd file) this has two side effects: - until now it was possible to replace "future instances" of an abstraction-object with externals. e.g. imagine that [foo] is resolved via ~/pd-externals/foo.pd. if someone installs a foo.pd_linux into ~/pd-externals/ during the Pd session, then the next time someone creates [foo] it will be the external! whether you like that behaviour or not, my implementation prevents it (however it is still possible to change the actual abstraction; e.g. if [foo] is resolved via /usr/lib/pd/extra/foo.pd and somebody saves a foo.pd into ~/pd-externals/, then the next time someone creates [foo] it will be the abstraction in ~/pd-externals/!) - since this means that we don't need to search the entire path for externals (that have not been there when we looked a second ago) whenever we create another instance of our abstraction, patches generally load faster (esp. when an abstraction is used often). the speedup is not tremendous, about 3x; but still...
canvas path iterator === to ease the implementation (and to avoid code duplication), i introduced an iterator, that will call a callback function for each searchpath of a canvas. this is now also used in canvas_open(), but might be useful elsewhere.
interactive use of declare === as a side effect (and not strictly related to the loading problem), i fixed [declare] so that creating a [declare -path foo] would allow you to *immediately* use the foo/ path. i always found it *very* annoying that you had to close/open the abstraction for [declare] to have an effect. (this functionality is disabled while the patch is currently /loading/)
not done === what i wanted to do as well, but haven't gotten around was proper canvas-local library loading, so you could do [declare -lib foo/bla]+[bla] in one abstraction and [declare -lib moo/bla]+[bla] in another abstraction and the two [bla] objectclasses would refer to different externals.
hmm, that mail is already quite long. and i have forgotten what else i wanted to say.
in any case, i would like to hear your opinion on all this.
in the meantime i'll submit the 4 patches of this implementation to the sf tracker.
gfamds IOhannes
¹ ah well, for certain use-cases one could disable the standard searchpaths altogether; not.