Hi everyone,
I've been working on some algorithms to eventually replace the way Pd finds and loads and creates objects. Sorry for the length of this mail, but you can skip some sections if you don't care about implementation details and/or user interface details :)
Table of Contents * Specifications * Example Loaders * Clearer Error Messages * Test Inputs * Test Outputs
Specifications --------------
As agreed by everyone so far, the new load order will be roughly:
for path in paths do -- the core does this bit for loader in loaders do loader(path, libray, object)
instead of the old, problematic:
for loader in loaders do for path in paths do -- the loader does this bit loader(path, library, object)
I restructured the loader plugin mechanism to be (pseudocode):
typedef err libloadfun(symbol *dir, symbol *lib); typedef err objloadfun(symbol *dir, symbol *lib, symbol *obj); struct loader { symbol *name; float priority; libloadfun *libfun; objloadfun *objfun; }
Example Loaders ---------------
-- binary loader is super simple Loader.new("bin", 70, function(dir, lib) return loadbin(dir, lib) end, function(dir, lib, obj) return loadbin(dir, obj) end)
-- directory loader is needed if there is no main library file -- the core code checks the results to see if it succeeded Loader.new("dir", 10, function(dir, lib) if dir:sub(-1 - #lib, -1) == ("/" .. lib) then return function() end else return nil, "dir doesn't match lib" end end, nil -- directory loader can't load objects )
-- the abstraction loader is quite complicated -- if the file exists it creates a callback that -- registers the class, with a constructor that -- runs the abstraction itself -- a Context here is essentially a 'canvas' Loader.new("txt", 30, nil, -- abstractions cannot register new classes function(dir, lib, obj) local src = dir .. "/" .. obj .. ".txt.lua" local f, err = fileexists(src) if f then local ldir, lobj = dir, obj return function() register(lobj, src, function(o, arguments) Context.push(Context.new({ ldir })) runtxt(ldir, lobj) Context.pop() end, src) end else return nil, err end end)
The current implementation is entirely in Lua, in an unpublished git repository. I'll upload it somewhere if there is any interest.
Clearer Error Messages ----------------------
The main upshot so far is clearer error messages instead of random possibly-working-but-perhaps-not implementation defined behaviour when more than one library defines the same name:
OBJECT [obj4] obj4 ... couldn't create (ambiguous) (found in: 'lib4') (found in: 'lib3') (found in: 'lib1') (found in: 'lib5') (found in: 'lib2') disambiguate by [import lib2/obj4] replacing lib2 by the library you want to import from
OBJECT [obj2] obj2 ... couldn't create (ambiguous qualified import) (imported: 'lib2/obj2') (imported: 'lib1/obj2') disambiguate by removing extra qualified imports
The error messages for objects that are not found are better too:
OBJECT [obj8] obj8 ... couldn't create (no such object) (looked in: 'lib4') (looked in: 'vanilla') (looked in: 'lib3') (looked in: 'lib1') (looked in: 'lib5') (looked in: 'lib2')
OBJECT [lib1/obj8] lib1/obj8 ... couldn't create (no such object in library)
OBJECT [lib6/obj1] lib6/obj1 ... couldn't create (no such library)
Note for that last [libc6/obj1] example: if libc6 actually did exist in the path but was not imported, the error would currently be the same. I want to make it so that [somelibrary/someobject] would work even when [somelibrary] is not imported (but such "implicit imports" would be deprecated, and warnings emitted).
Test Inputs -----------
The test input directory structure (.bin.lua approximates .pd_linux etc, .txt.lua approximates .pd patches/abstractions): ----8<---- .: global prj1
./global: lib1 lib2 lib3 lib3.bin.lua lib4 lib5.bin.lua vanilla
./global/lib1: obj1.txt.lua obj3.txt.lua obj4.txt.lua obj5.txt.lua obj7.txt.lua obj2.txt.lua obj4.bin.lua obj5.bin.lua obj6.bin.lua
./global/lib2: lib2.bin.lua obj2.txt.lua obj4.bin.lua obj5.bin.lua obj6.bin.lua obj1.txt.lua obj3.txt.lua obj4.txt.lua obj5.txt.lua obj7.txt.lua
./global/lib3: obj1.txt.lua obj3.txt.lua obj4.txt.lua obj5.txt.lua obj7.txt.lua obj2.txt.lua obj4.bin.lua obj5.bin.lua obj6.bin.lua
./global/lib4: obj1.txt.lua obj3.txt.lua obj4.txt.lua obj5.txt.lua obj7.txt.lua obj2.txt.lua obj4.bin.lua obj5.bin.lua obj6.bin.lua
./global/vanilla: debug.bin.lua
./prj1: prj1.txt.lua ----8<----
The test input file (prj1.txt.lua) imports stuff and creates objects: ----8<---- import({ "lib1", "lib1/obj1", "lib1/obj2", "lib2", "lib2/obj2", "lib2/obj3", "lib3", "lib4", "lib5" }) for o = 1,8 do object("obj" .. o) end for l = 1,6 do for o = 1,8 do object("lib" .. l .. "/" .. "obj" .. o) end end ----8<----
Test Outputs ------------
The full raw output: http://claudiusmaximus.goto10.org/t/pd/pd-new-loader-test-4-verbose.txt
The output with the STAT lines filtered out: http://claudiusmaximus.goto10.org/t/pd/pd-new-loader-test-4-quiet.txt
The output with the STAT lines sorted and duplicates counted: http://claudiusmaximus.goto10.org/t/pd/pd-new-loader-test-4-statistics.txt (The counts are low because the results are cached, this will need to be changed for live-coding abstractions etc...)
Hopefully I'm heading in the right direction with this, if not then yell loudly!
Thanks,
Claude