When 0.39 begins to wane (so [declare] can be used), ...
Careful here: [declare -path ...] is disabled inside of abstractions in Pd-0.41.
Right -- but [declare -path ...] is terribly useful for not having a patch's main directory cluttered with 100 abstractions, which was the main point... but since 0.39 is still widely in use I tend to avoid it unless it's for patches I know only I am going to be running. I guess it's okay to be conservative in some parts of life. =o)
OT -- out of curiosity, if it were to be enabled within an abstraction, would the -path be relative to the abstraction file or to the patch in which it's instantiated?
M
On Jul 29, 2008, at 4:29 PM, Matt Barber wrote:
When 0.39 begins to wane (so [declare] can be used), ...
Careful here: [declare -path ...] is disabled inside of abstractions in Pd-0.41.
Right -- but [declare -path ...] is terribly useful for not having a patch's main directory cluttered with 100 abstractions, which was the main point... but since 0.39 is still widely in use I tend to avoid it unless it's for patches I know only I am going to be running. I guess it's okay to be conservative in some parts of life. =o)
OT -- out of curiosity, if it were to be enabled within an abstraction, would the -path be relative to the abstraction file or to the patch in which it's instantiated?
The relative path stuff seems to be a can of worms that is more
trouble that it is worth, IMHO. But in order to make sure it doesn't
make things more complicated, the path declared by [declare -path]
would have to be same absolute path everywhere it is used.
I think the simplest solution is to make the only option be loading
libraries and setting the global search path. Then use a unified
library format that is very easy to setup. That's the goal with libdir.
.hc
M
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/ listinfo/pd-list
I have the audacity to believe that peoples everywhere can have three
meals a day for their bodies, education and culture for their minds,
and dignity, equality and freedom for their spirits. - Martin
Luther King, Jr.
Hallo, Matt Barber hat gesagt: // Matt Barber wrote:
When 0.39 begins to wane (so [declare] can be used), ...
Careful here: [declare -path ...] is disabled inside of abstractions in Pd-0.41.
Right -- but [declare -path ...] is terribly useful for not having a patch's main directory cluttered with 100 abstractions, which was the main point... but since 0.39 is still widely in use I tend to avoid it unless it's for patches I know only I am going to be running. I guess it's okay to be conservative in some parts of life. =o)
OT -- out of curiosity, if it were to be enabled within an abstraction, would the -path be relative to the abstraction file or to the patch in which it's instantiated?
Relative paths are important if you use declare in the way you described here: to make shipping "bundles" easier by putting all used abstractions into a subdirectory and adding that subdir to the searchpath of the MAIN.pd. As you don't know where a user puts your project, you cannot use absolute paths.
Now inside of an abstraction I would discern two use cases: One is using declare just as in the MAIN patch to add further subdirectories for other helper abstractions. Here the only thing that makes sense IMO when using relative paths is to have them relative to the abstraction itself.
Another use case is the path for "resources" like soundfiles or sequencer patterns. The example Miller once mentioned on pd-dev was having a [soundfiler] in a sample player abstraction. Usually this abstraction would live somewhere far away from your soundfiles or your MAIN.pd.
[soundfiler] also looks for soundfiles in the pd search path, so users may feel inclined to manipulate the path to make the sampler abstraction find wav-files in a "snd"-directory somewhere without giving the full absolute path. Here it probably is more common to use paths relative to MAIN.pd instead of sampleplayer.pd so the MAIN-file's -path setting would reach into the child patch as well.
A drum synth abstraction may be different again: Maybe it has some default samples for kick and hihat in a subdirectory next to itself. Now you could manipulate the path to make it find these samples first. Again absolute paths don't work here, as your synth.pd may be installed anywhere.
As there is no consensus which of the two path alternatives should be used inside an abstraction, -path currently is disabled completely in an abstraction. (Btw. I don't know how an abstraction knows that it's used as an abstraction, not as MAIN.pd.)
Ciao
Frank Barknecht _ ______footils.org__
On Jul 30, 2008, at 4:01 AM, Frank Barknecht wrote:
Hallo, Matt Barber hat gesagt: // Matt Barber wrote:
When 0.39 begins to wane (so [declare] can be used), ...
Careful here: [declare -path ...] is disabled inside of abstractions in Pd-0.41.
Right -- but [declare -path ...] is terribly useful for not having a patch's main directory cluttered with 100 abstractions, which was the main point... but since 0.39 is still widely in use I tend to
avoid it unless it's for patches I know only I am going to be running. I
guess it's okay to be conservative in some parts of life. =o)OT -- out of curiosity, if it were to be enabled within an abstraction, would the -path be relative to the abstraction file
or to the patch in which it's instantiated?Relative paths are important if you use declare in the way you described here: to make shipping "bundles" easier by putting all used abstractions into a subdirectory and adding that subdir to the searchpath of the MAIN.pd. As you don't know where a user puts your project, you cannot use absolute paths.
Now inside of an abstraction I would discern two use cases: One is using declare just as in the MAIN patch to add further subdirectories for other helper abstractions. Here the only thing that makes sense IMO when using relative paths is to have them relative to the abstraction itself.
Another use case is the path for "resources" like soundfiles or sequencer patterns. The example Miller once mentioned on pd-dev was having a [soundfiler] in a sample player abstraction. Usually this abstraction would live somewhere far away from your soundfiles or your MAIN.pd.
[soundfiler] also looks for soundfiles in the pd search path, so users may feel inclined to manipulate the path to make the sampler abstraction find wav-files in a "snd"-directory somewhere without giving the full absolute path. Here it probably is more common to use paths relative to MAIN.pd instead of sampleplayer.pd so the MAIN-file's -path setting would reach into the child patch as well.
A drum synth abstraction may be different again: Maybe it has some default samples for kick and hihat in a subdirectory next to itself. Now you could manipulate the path to make it find these samples first. Again absolute paths don't work here, as your synth.pd may be installed anywhere.
As there is no consensus which of the two path alternatives should be used inside an abstraction, -path currently is disabled completely in an abstraction. (Btw. I don't know how an abstraction knows that it's used as an abstraction, not as MAIN.pd.)
There is consensus if we consider how other programming languages
have done it. From what I have seen in Python, Java, and C (and
probably many others) there are two options: global and local-to-the-
file.
If we want to have namespaces working, then I suggest we use the
tried-n-true model of other languages. If people are really
interested in experimenting with ideas of how to handle namespaces,
then we could try other options. Either way, we should start by
getting the tried-n-true model working first, IMHO. Then if people
want to experiment with things like inheriting namespaces from the
parent, there will be a solid, working foundation to build upon.
I plan on working on this in depth before the end of the year.
.hc
Ciao
Frank Barknecht _
______footils.org__
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/ listinfo/pd-list
Access to computers should be unlimited and total. - the hacker ethic
Hallo, Hans-Christoph Steiner hat gesagt: // Hans-Christoph Steiner wrote:
There is consensus if we consider how other programming languages
have done it.
That sounds a bit like: there is consensus if we drop all other opinions.
I'm joking here and smiling very brightly! ;) :-)
Ciao
Frank
Hans-Christoph Steiner wrote:
If we want to have namespaces working, then I suggest we use the
tried-n-true model of other languages. If people are really
interested in experimenting with ideas of how to handle namespaces,
then we could try other options. Either way, we should start by
getting the tried-n-true model working first, IMHO. Then if people
want to experiment with things like inheriting namespaces from the
parent, there will be a solid, working foundation to build upon.
what's the tried-n-true model? and why is expanding namespaces to (sub)classes not part of tried-n-true? I think an inheritance model should be introduced into pd.
declare is intended to load libraries or objectclasses only locally (only available to the patch that holds the declare object - since 0.41 this should be the case.)
for nested abstractions there are three possibilities: (1) don't inherit anything, (2) inherit from parent to child, (3) inherit from child to parent. (or several declare options that allow all three possibilities.)
I would like to "vote" for solution 2. but I think there are technical problems with this option: afaik it is not possible to "overwrite" a declaration. for now it is first come first serve. so if an abstraction inside a nested patch wanted to use a certain objectclass which is in conflict with the parent patch, that would just not be possible. and another problem seems to be that abstractions get loaded first, so the first come - first serve works even worse right now, because it loads declarations of abstractions before the parent patch. please correct me, if I am wrong.
I see usecases for method (3), too. if you want to throw a certain set of declarations into all your patches as abstractions. but I think this causes more problems than it solves. what if you have two conflicting abstractions in parallel, that both try to set the namespace for the parent patch?
marius.
On Jul 30, 2008, at 3:15 PM, marius schebella wrote:
Hans-Christoph Steiner wrote:
If we want to have namespaces working, then I suggest we use the
tried-n-true model of other languages. If people are really
interested in experimenting with ideas of how to handle
namespaces, then we could try other options. Either way, we
should start by getting the tried-n-true model working first,
IMHO. Then if people want to experiment with things like
inheriting namespaces from the parent, there will be a solid,
working foundation to build upon.what's the tried-n-true model? and why is expanding namespaces to
(sub)classes not part of tried-n-true? I think an inheritance model
should be introduced into pd.declare is intended to load libraries or objectclasses only locally
(only available to the patch that holds the declare object - since
0.41 this should be the case.)for nested abstractions there are three possibilities: (1) don't
inherit anything, (2) inherit from parent to child, (3) inherit
from child to parent. (or several declare options that allow all
three possibilities.)I would like to "vote" for solution 2. but I think there are
technical problems with this option: afaik it is not possible to
"overwrite" a declaration. for now it is first come first serve. so
if an abstraction inside a nested patch wanted to use a certain
objectclass which is in conflict with the parent patch, that would
just not be possible. and another problem seems to be that abstractions get loaded first,
so the first come - first serve works even worse right now, because
it loads declarations of abstractions before the parent patch. please correct me, if I am wrong.I see usecases for method (3), too. if you want to throw a certain
set of declarations into all your patches as abstractions. but I
think this causes more problems than it solves. what if you have
two conflicting abstractions in parallel, that both try to set the
namespace for the parent patch?marius.
I think it is essential that each object be complete and functional
no matter what other things are setup in the namespace. Having one
patch inherit the namespace of the other could cause an object to
break if a different [prepend] was loaded first, for example.
In order for objectclasses to be self-contained, there needs to be a
canvas-local namespace for every patch that is checked before any
inherited namespace. That is missing right now. There could then be
a "parent" namespace, then there is already the global namespace.
Personally, I think this is overly complicated. I think that global
and canvas-local is enough. Those two are the essential ones.
Python, for example, does have more levels, like the "module" level
namespace, which could be seen as a kind of parallel to a "parent"
namespace. As long as there is a true canvas-local namespace, then
this is a workable option.
.hc
If nature has made any one thing less susceptible than all others of
exclusive property, it is the action of the thinking power called an
idea, which an individual may exclusively possess as long as he keeps
it to himself; but the moment it is divulged, it forces itself into
the possession of everyone, and the receiver cannot dispossess
himself of it. - Thomas Jefferson
Hans-Christoph Steiner wrote:
On Jul 30, 2008, at 3:15 PM, marius schebella wrote:
Hans-Christoph Steiner wrote:
If we want to have namespaces working, then I suggest we use the
tried-n-true model of other languages. If people are really
interested in experimenting with ideas of how to handle namespaces,
then we could try other options. Either way, we should start by
getting the tried-n-true model working first, IMHO. Then if people
want to experiment with things like inheriting namespaces from the
parent, there will be a solid, working foundation to build upon.what's the tried-n-true model? and why is expanding namespaces to (sub)classes not part of tried-n-true? I think an inheritance model should be introduced into pd.
declare is intended to load libraries or objectclasses only locally (only available to the patch that holds the declare object - since 0.41 this should be the case.)
for nested abstractions there are three possibilities: (1) don't inherit anything, (2) inherit from parent to child, (3) inherit from child to parent. (or several declare options that allow all three possibilities.)
I would like to "vote" for solution 2. but I think there are technical problems with this option: afaik it is not possible to "overwrite" a declaration. for now it is first come first serve. so if an abstraction inside a nested patch wanted to use a certain objectclass which is in conflict with the parent patch, that would just not be possible. and another problem seems to be that abstractions get loaded first, so the first come - first serve works even worse right now, because it loads declarations of abstractions before the parent patch. please correct me, if I am wrong.
I see usecases for method (3), too. if you want to throw a certain set of declarations into all your patches as abstractions. but I think this causes more problems than it solves. what if you have two conflicting abstractions in parallel, that both try to set the namespace for the parent patch?
marius.
I think it is essential that each object be complete and functional no matter what other things are setup in the namespace. Having one patch inherit the namespace of the other could cause an object to break if a different [prepend] was loaded first, for example.
In order for objectclasses to be self-contained, there needs to be a canvas-local namespace for every patch that is checked before any inherited namespace. That is missing right now. There could then be a "parent" namespace, then there is already the global namespace.
Personally, I think this is overly complicated. I think that global and canvas-local is enough. Those two are the essential ones.Python, for example, does have more levels, like the "module" level namespace, which could be seen as a kind of parallel to a "parent" namespace. As long as there is a true canvas-local namespace, then this is a workable option.
.hc
hi, I realized that the example I gave in the beginning of this thread was bad, because it did not use declare at all! And reading also the discussion on the similar thread (loooooooong) clarified some things. My intention is still to give patches a better structure for the future. One of the things that changed my thinking was the fact that there is a difference between namespaces and searchpaths. If local patch/canvas namespaces are working, the whole "loading before" and "loading after" would become irrelevant. relying only on searchpaths means everything is dependend on the loading order of libraries, or object classes respectively. but include, define, require, declare, import, loadlib, loadpath all have slightly different meanings... It would be cool, if the behaviour would be better pointed out; the current and the future one, too. that would make the whole mechanism more clear, comprehensible and traceable. that said, back to the problem of declaring relative search paths in abstractions. the idea of adding a relative path is that instead of writing path/to/my/folder/someresource1.xyz one could write [declare -path path/to/my/folder] and then references someresource1.xyz without the need to type the path over and over again. that is useful for resources like images or soundfiles, (the question whether this should also work for abstractions too, is still undecided). situation 1: the abstraction depends on resorces that are relative to the abstraction directory. no question this situation will appear and can only be managed, if relative paths are relative to the file location of the abstraction. situation 2: An abstraction wants to access a folder that is relative to the file location of the pd-patch file that nests the abstraction. for example a slideshow-abstraction that should grab 20 images of a folder relative to the parent patch. in this case [declare -path $1] inside the abstraction would try to find the folder relative to the abstraction location instead of the "parent" patch loacation. This is already the case if sending a relative path to an object in an abstraction like [open path/to/resource.xyz(, which is/was always relative to the file loaction of the abstraction, even if you use [open $1( and feed the path from outside. so after all, I guess Reverend Roman was right. the solution for situation 2 would be to put the declare object in the patchfile to which the directory is relative to. cheers, marius.
Hallo, I'm just adding my voice to the choir here before falling asleep after playing with this in my setup tonight - I definitely think [declare -path] should be re-enabled inside abstractions, and should be relative to the abstraction's file location.
My goal is to remove all directories from the global path because I discovered the ~40 that come with Pd-extended plus the other 15 I added personally were playing a very significant role in the rapidly increasing (now ~15 minute) loadtime of my large performance patch.
My layout (in a cartoon rendering) is something like this: Drum-modules/ Datastructure-abs/ MIDI-abs/ Helper-abs/ Performance-patches/
If I'd like to load DrumModules abstractions from within a Performance-patch (with [declare -path ../Drum-modules]), and DrumModules abstractions use Helper-abs abstractions, I need [declare -path ../Helper-abs] to work within a Drum-modules abstraction.
If no Helper-abs are used within Performance-patch, then I don't want to put [declare -path ../Helper-abs] there, because it would mean Performance-patch would be looking in Helper-abs when all I want are Drum-modules, slowing down the search process.*
Is it true that having more files in any one path increases the lookup time? I'd think it a good thing to encourage breaking large patch collections into folders for organization reasons alone, and if there are performance benefits to that too, then all the better!
Best Luke
*and, to drive the point home further, say I also had "Score-modules" inside Performance-patch that needed MIDI-abs and Datastructure-abs: I /really/ don't want to put [declare -path ../MIDI-abs -path ../Datastructure-abs] in Performance-patch because now Performance-patch is looking for Drum-modules in MIDI-abs and Datastructure-abs for no reason.
Worse, Drum-modules inherit these paths too and are looking fruitlessly for their Helper-abs in MIDI-abs and Datastructure-abs, which is what I was trying to avoid in the first place when switching from global to local namespaces.
On Sun, Aug 3, 2008 at 12:33 PM, marius schebella marius.schebella@gmail.com wrote:
Hans-Christoph Steiner wrote:
On Jul 30, 2008, at 3:15 PM, marius schebella wrote:
Hans-Christoph Steiner wrote:
If we want to have namespaces working, then I suggest we use the tried-n-true model of other languages. If people are really interested in experimenting with ideas of how to handle namespaces, then we could try other options. Either way, we should start by getting the tried-n-true model working first, IMHO. Then if people want to experiment with things like inheriting namespaces from the parent, there will be a solid, working foundation to build upon.
what's the tried-n-true model? and why is expanding namespaces to (sub)classes not part of tried-n-true? I think an inheritance model should be introduced into pd.
declare is intended to load libraries or objectclasses only locally (only available to the patch that holds the declare object - since 0.41 this should be the case.)
for nested abstractions there are three possibilities: (1) don't inherit anything, (2) inherit from parent to child, (3) inherit from child to parent. (or several declare options that allow all three possibilities.)
I would like to "vote" for solution 2. but I think there are technical problems with this option: afaik it is not possible to "overwrite" a declaration. for now it is first come first serve. so if an abstraction inside a nested patch wanted to use a certain objectclass which is in conflict with the parent patch, that would just not be possible. and another problem seems to be that abstractions get loaded first, so the first come - first serve works even worse right now, because it loads declarations of abstractions before the parent patch. please correct me, if I am wrong.
I see usecases for method (3), too. if you want to throw a certain set of declarations into all your patches as abstractions. but I think this causes more problems than it solves. what if you have two conflicting abstractions in parallel, that both try to set the namespace for the parent patch?
marius.
I think it is essential that each object be complete and functional no matter what other things are setup in the namespace. Having one patch inherit the namespace of the other could cause an object to break if a different [prepend] was loaded first, for example.
In order for objectclasses to be self-contained, there needs to be a canvas-local namespace for every patch that is checked before any inherited namespace. That is missing right now. There could then be a "parent" namespace, then there is already the global namespace. Personally, I think this is overly complicated. I think that global and canvas-local is enough. Those two are the essential ones.
Python, for example, does have more levels, like the "module" level namespace, which could be seen as a kind of parallel to a "parent" namespace. As long as there is a true canvas-local namespace, then this is a workable option.
.hc
hi, I realized that the example I gave in the beginning of this thread was bad, because it did not use declare at all! And reading also the discussion on the similar thread (loooooooong) clarified some things. My intention is still to give patches a better structure for the future. One of the things that changed my thinking was the fact that there is a difference between namespaces and searchpaths. If local patch/canvas namespaces are working, the whole "loading before" and "loading after" would become irrelevant. relying only on searchpaths means everything is dependend on the loading order of libraries, or object classes respectively. but include, define, require, declare, import, loadlib, loadpath all have slightly different meanings... It would be cool, if the behaviour would be better pointed out; the current and the future one, too. that would make the whole mechanism more clear, comprehensible and traceable. that said, back to the problem of declaring relative search paths in abstractions. the idea of adding a relative path is that instead of writing path/to/my/folder/someresource1.xyz one could write [declare -path path/to/my/folder] and then references someresource1.xyz without the need to type the path over and over again. that is useful for resources like images or soundfiles, (the question whether this should also work for abstractions too, is still undecided). situation 1: the abstraction depends on resorces that are relative to the abstraction directory. no question this situation will appear and can only be managed, if relative paths are relative to the file location of the abstraction. situation 2: An abstraction wants to access a folder that is relative to the file location of the pd-patch file that nests the abstraction. for example a slideshow-abstraction that should grab 20 images of a folder relative to the parent patch. in this case [declare -path $1] inside the abstraction would try to find the folder relative to the abstraction location instead of the "parent" patch loacation. This is already the case if sending a relative path to an object in an abstraction like [open path/to/resource.xyz(, which is/was always relative to the file loaction of the abstraction, even if you use [open $1( and feed the path from outside. so after all, I guess Reverend Roman was right. the solution for situation 2 would be to put the declare object in the patchfile to which the directory is relative to. cheers, marius.
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Hallo, Hans-Christoph Steiner hat gesagt: // Hans-Christoph Steiner wrote:
There is consensus if we consider how other programming languages
have done it. From what I have seen in Python, Java, and C (and
probably many others) there are two options: global and local-to-the- file.
Okay, now a more serious, but very long and sometimes shaky answer.
As I see it we're dealing with some intertwined, but different issues here, and also there still is too much guesswork involved, where we should develop a concrete and detailed specification. The only docs in this direction that I could find on pd.info are these two by you: http://puredata.info/dev/PdNamespaces http://puredata.info/docs/developer/LibdirFormatForLibrariesOfObjects
These should be fleshed out and become much more detailed, i.e. with concrete examples and definitions of terms. Otherwise we will risk to talk at cross-purposes. For example, even the sentences "We should get namespaces working" or "Lets do as Lua did" are very ambigous if we look closely.
To get back to the intertwined issues I see:
One is the actual namespace which I for now would define as
A namespace is the set of class names which are available to be converted into concrete objects when typed into an object box in a patch canvas.
Note that this is not that thing that we should "get working", it's just a try at a definition of the term. A try at what "get working" might mean could be this:
Pd should be enabled to modify the contents of the namespaces in ways (sic!) that make everyone happy.
We then also have another issue which is "The Path". I don't have a good name for this yet. In my previous mails I often called it "-path" to hint at the command line option which sets it. [declare] modifies "The Path", which in turn modifies the namespace currently (because Pd works that way. This isn't a requirement as Lua or Python modify namespaces in a completly different way). "The Path" in Pd also is responsible for other ressources like [soundfiler]'s path. (Idea: We could introduce a Ressource Path to separate the class search path and other search paths like for soundfiler.)
A relative of The Path, but not the same, is "The Helppath", which lets objects call help files. You all know it.
Let me stress again, that in my view Paths are not equivalent to the namespace and they are not the only way to modify the namespace, see below. In Python and Lua they just are values in variables (e.g. "package.path" and "package.cpath" in Lua).
The confusion about The Path and The Namespace comes from the fact, that in Pd currently one (even the only one for abstractions) way to modify the contents of the namespace is to modify The Path. As we all know this has side effects, for example modifying The Path also modifies what [soundfiler] sees.
But there are other ways to modify the contents of the namespace, not just by modifying The Path. One is to load a library of multiclass objects, another is to use certain loaders. I really, really like IOhannes' approach of moving the loading of abstractions into a loader (Patch #1990686) as it may open up a new way of thinking about namespaces separate from just modifying The Path (which I fear may lead into a dead end if things turn out bad ...)
Finally there are two other, more complicated concepts to maybe consider, namely "The Environment" and "The Scope". I refer to concepts of traditional programming languages here. For Environment see the SCIP for example: http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-10.html#%_sec_1.1.2 http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-21.html#%_sec_3.2
For Environments in Lua check this: http://www.lua.org/pil/14.html
Scope in Pd so far is easy: Everything is global, even $0-variables. Locality is expressed in patch cords, which are as local as local gets (except maybe in Gem). [declare] and [import] showed a possible different way of locality: canvas-locality. It's an open question if this should extend to subpatches and abstractions. (In Lua, local scope can extend into deeper regions: see non-local variables.)
The other languages mentioned have very elaborate scoping rules. Lua is a good example as it defaults to a global scope (like Pd) as well, while still providing the "local" keyword to make things local to a block, and it even has something in between ("non-local variables" or "upvalues"). In Python it is the other way around, stuff is local as default.
Environments are crucial for the implementation of namespaces in Lua, as "importing" something means changing the global environment. In Pd I have no idea how to express the concept of an Environment (maybe language theory experts like Matju or A. Graef can help me here?) but I guess, the fact alone, that Lua uses them for importing, means, that we may need to find a different way when doing imports from how Lua does it.
In the end to me means this means that this:
If we want to have namespaces working, then I suggest we use the
tried-n-true model of other languages.
isn't as easy as it sounds. Not by far.
Well, sorry for this long and in parts very confusing/confused mail. I myself am trying to get my head around these issues, so you see me live coding here and please apologize any inconsistencies. But I really think, that there's a time now to lean back for a moment, take a deep breath and enjoy the whole scenery to then discuss and see where we are standing and what solution would be a good long term solution. To consider advantages and disadvantages and bring a bit of order and some specifications into all these problems.
Oh, and maybe this should move to pd-dev?
Ciao
Frank Barknecht _ ______footils.org__
On Jul 30, 2008, at 3:35 PM, Frank Barknecht wrote:
Hallo, Hans-Christoph Steiner hat gesagt: // Hans-Christoph Steiner wrote:
There is consensus if we consider how other programming languages have done it. From what I have seen in Python, Java, and C (and probably many others) there are two options: global and local-to- the- file.
Okay, now a more serious, but very long and sometimes shaky answer.
As I see it we're dealing with some intertwined, but different issues here, and also there still is too much guesswork involved, where we should develop a concrete and detailed specification. The only docs in this direction that I could find on pd.info are these two by you: http://puredata.info/dev/PdNamespaces http://puredata.info/docs/developer/LibdirFormatForLibrariesOfObjects
These should be fleshed out and become much more detailed, i.e. with concrete examples and definitions of terms. Otherwise we will risk to talk at cross-purposes. For example, even the sentences "We should get namespaces working" or "Lets do as Lua did" are very ambigous if we look closely.
To get back to the intertwined issues I see:
One is the actual namespace which I for now would define as
A namespace is the set of class names which are available to be converted into concrete objects when typed into an object box in a patch canvas.
I edited this and added it to the PdNamespaces wiki.
http://puredata.info/dev/PdNamespaces
Note that this is not that thing that we should "get working", it's just a try at a definition of the term. A try at what "get working" might mean could be this:
Pd should be enabled to modify the contents of the namespaces in ways (sic!) that make everyone happy.
We then also have another issue which is "The Path". I don't have a good name for this yet. In my previous mails I often called it "-path" to hint at the command line option which sets it. [declare] modifies "The Path", which in turn modifies the namespace currently (because Pd works that way. This isn't a requirement as Lua or Python modify namespaces in a completly different way). "The Path" in Pd also is responsible for other ressources like [soundfiler]'s path. (Idea: We could introduce a Ressource Path to separate the class search path and other search paths like for soundfiler.)
A relative of The Path, but not the same, is "The Helppath", which lets objects call help files. You all know it.
Let me stress again, that in my view Paths are not equivalent to the namespace and they are not the only way to modify the namespace, see below. In Python and Lua they just are values in variables (e.g. "package.path" and "package.cpath" in Lua).
The confusion about The Path and The Namespace comes from the fact, that in Pd currently one (even the only one for abstractions) way to modify the contents of the namespace is to modify The Path. As we all know this has side effects, for example modifying The Path also modifies what [soundfiler] sees.
It seems to me that using the canvas-local and global paths for
everything that opens a file isn't really a good idea. You never
know where a file could come from. But that is entrenched, so it is
not going anywhere. Personally I think [soundfiler] should only look
in the current directory of the patch if it is a relative path. If
you need to use more complicated paths, you can use [getdir]. But I
think we should let this issue rest and just focus on the namespace
for loading objectclasses.
But there are other ways to modify the contents of the namespace, not just by modifying The Path. One is to load a library of multiclass objects, another is to use certain loaders. I really, really like IOhannes' approach of moving the loading of abstractions into a loader (Patch #1990686) as it may open up a new way of thinking about namespaces separate from just modifying The Path (which I fear may lead into a dead end if things turn out bad ...)
Finally there are two other, more complicated concepts to maybe consider, namely "The Environment" and "The Scope". I refer to concepts of traditional programming languages here. For Environment see the SCIP for example: http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-10.html#% _sec_1.1.2 http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-21.html#%_sec_3.2
For Environments in Lua check this: http://www.lua.org/pil/14.html
Scope in Pd so far is easy: Everything is global, even $0-variables. Locality is expressed in patch cords, which are as local as local gets (except maybe in Gem). [declare] and [import] showed a possible different way of locality: canvas-locality. It's an open question if this should extend to subpatches and abstractions. (In Lua, local scope can extend into deeper regions: see non-local variables.)
The other languages mentioned have very elaborate scoping rules. Lua is a good example as it defaults to a global scope (like Pd) as well, while still providing the "local" keyword to make things local to a block, and it even has something in between ("non-local variables" or "upvalues"). In Python it is the other way around, stuff is local as default.
Environments are crucial for the implementation of namespaces in Lua, as "importing" something means changing the global environment. In Pd I have no idea how to express the concept of an Environment (maybe language theory experts like Matju or A. Graef can help me here?) but I guess, the fact alone, that Lua uses them for importing, means, that we may need to find a different way when doing imports from how Lua does it.
The Lua-style "environment" in Pd is basically just the global path
and the global helppath. There aren't really any other global
variables that I can think of.
In the end to me means this means that this:
If we want to have namespaces working, then I suggest we use the tried-n-true model of other languages.
isn't as easy as it sounds. Not by far.
Well, sorry for this long and in parts very confusing/confused mail. I myself am trying to get my head around these issues, so you see me live coding here and please apologize any inconsistencies. But I really think, that there's a time now to lean back for a moment, take a deep breath and enjoy the whole scenery to then discuss and see where we are standing and what solution would be a good long term solution. To consider advantages and disadvantages and bring a bit of order and some specifications into all these problems.
Oh, and maybe this should move to pd-dev?
That's what this wiki page is for, let's start documenting ideas and
cataloging behaviors. This is not such a hard problem. It just
needs some careful cataloging of the possibilities and a little
research into how others have solved this issues. Most of the work
will be implementing it.
http://puredata.info/dev/PdNamespaces
.hc
Ciao
Frank Barknecht _
______footils.org__
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/ listinfo/pd-list
News is what people want to keep hidden and everything else is
publicity. - Bill Moyers
Hallo, Hans-Christoph Steiner hat gesagt: // Hans-Christoph Steiner wrote:
The Lua-style "environment" in Pd is basically just the global path
and the global helppath.
Maybe. But Lua also does have a path variable ("package.path") which is very similar to Pd's -path as it's used to specify where to search for modules.
package.path however is practically never modified and stays the way it is through the lifetime of a program. The whole namespace/enviroment manipulation happens elsewhere. By separating searchpath and namespace manipulation Lua gains a lot more flexibility than Pd has.
To illustrate this a bit more: In Lua, the function "require" loads modules from the searchpath and then can manipulate the namespace (Actually "require" doesn't do any namespace manipulation itself, instead that's in the responsibility of the module loaded.) "require" is the *only* function that actually accesses the package.path variable. package.path is generally unused everywhere else.
In Pd, the global path is used to directly manipulate the namespace. This is much more limited, full of side-effects and for example is the only reason that multi-class libraries had to be deprecated in pd-extended - which still ships some of these: Gem and PDP.
I'm not yet sure what model a better namespace manipulator could follow, but with the loader infrastructure, we maybe have all the hooks already in place to make a better namespace modifier that doesn't use the path in the old way. For example such a loader could be made to load Gem with an object prefix (whatever use this may have)
Ciao
Frank Barknecht
Hans-Christoph Steiner wrote:
It seems to me that using the canvas-local and global paths for
everything that opens a file isn't really a good idea. You never
know where a file could come from. But that is entrenched, so it is
not going anywhere.
but isn't this right now?
when a files gets searched it will be searched "everywhere" (in The Path); in Pd-vanilla (default settings), it might come from "." (wherever that is) or from <pd>/extra/ in Pd-extended (default settings on OSX) it will additionally search them in 41 other locations.
these numbers come from [soundfiler] opening a (non-existant) sound-file directly in the parent patch.
the example is probably a good argument against global paths, but i don't think adding canvas-local paths will make it worse. the important thing is that there is a logic behind the search order (that is comprehendable by humans)
Personally I think [soundfiler] should only look
in the current directory of the patch if it is a relative path. If
otoh, i do think that frank's and marius' request for abstraction-local paths are valid and important.
you need to use more complicated paths, you can use [getdir]. But I
unfortunately i don't have [getdir]. proposing an external sounds pretty ugly to me, when it comes to a fundamental problem. of course we could all use not Pd.
think we should let this issue rest and just focus on the namespace
for loading objectclasses.
yes, i think that is what frank was trying to say: due to the missing definitions we keep mixing up searchpaths and namespaces - they are different things (only Pd happens to make them related; right, java does so too, but who uses java?)
anyhow, i very much like frank's idea of separating the object searchpath from the ressourcespath. (but then it is nice to have them unified for special applications; e.g. when i want to tread a Pd-patch as data)
and btw, i always found that the entire helppath thing was a ugly and should be removed as soon as possible.
Oh, and maybe this should move to pd-dev?
That's what this wiki page is for, let's start documenting ideas and
well, personally i feel more inclined to follow a discussion (and participate in it) on push media rather than pull media.
mgadrt IOhannes
IOhannes m zmoelnig schreef:
yes, i think that is what frank was trying to say: due to the missing definitions we keep mixing up searchpaths and namespaces - they are different things (only Pd happens to make them related; right, java does so too, but who uses java?)
i use java sometimes, but java's path<->namespace correspondency is not something that should be emulated, IMO. it's ugly.
well, personally i feel more inclined to follow a discussion (and participate in it) on push media rather than pull media.
ditto
Hallo, Damian Stewart hat gesagt: // Damian Stewart wrote:
IOhannes m zmoelnig schreef:
well, personally i feel more inclined to follow a discussion (and participate in it) on push media rather than pull media.
ditto
ditto++, but after or at certain points during the discussion of course some results should go to the wiki. But it's still a bit early IMO.
Ciao
Frank Barknecht _ ______footils.org__
On Wed, 2008-07-30 at 10:01 +0200, Frank Barknecht wrote:
Hallo, Matt Barber hat gesagt: // Matt Barber wrote:
When 0.39 begins to wane (so [declare] can be used), ...
Careful here: [declare -path ...] is disabled inside of abstractions in Pd-0.41.
Right -- but [declare -path ...] is terribly useful for not having a patch's main directory cluttered with 100 abstractions, which was the main point... but since 0.39 is still widely in use I tend to avoid it unless it's for patches I know only I am going to be running. I guess it's okay to be conservative in some parts of life. =o)
OT -- out of curiosity, if it were to be enabled within an abstraction, would the -path be relative to the abstraction file or to the patch in which it's instantiated?
Relative paths are important if you use declare in the way you described here: to make shipping "bundles" easier by putting all used abstractions into a subdirectory and adding that subdir to the searchpath of the MAIN.pd. As you don't know where a user puts your project, you cannot use absolute paths.
Now inside of an abstraction I would discern two use cases: One is using declare just as in the MAIN patch to add further subdirectories for other helper abstractions. Here the only thing that makes sense IMO when using relative paths is to have them relative to the abstraction itself.
Another use case is the path for "resources" like soundfiles or sequencer patterns. The example Miller once mentioned on pd-dev was having a [soundfiler] in a sample player abstraction. Usually this abstraction would live somewhere far away from your soundfiles or your MAIN.pd.
[soundfiler] also looks for soundfiles in the pd search path, so users may feel inclined to manipulate the path to make the sampler abstraction find wav-files in a "snd"-directory somewhere without giving the full absolute path. Here it probably is more common to use paths relative to MAIN.pd instead of sampleplayer.pd so the MAIN-file's -path setting would reach into the child patch as well.
A drum synth abstraction may be different again: Maybe it has some default samples for kick and hihat in a subdirectory next to itself. Now you could manipulate the path to make it find these samples first. Again absolute paths don't work here, as your synth.pd may be installed anywhere.
As there is no consensus which of the two path alternatives should be used inside an abstraction, -path currently is disabled completely in an abstraction. (Btw. I don't know how an abstraction knows that it's used as an abstraction, not as MAIN.pd.)
i question, if someone is seriously voting for the 'relative to parent' approach, since it is full of troubles:
- it's not clear, what the parent is: is it the top-most patch or the
patch, that is the direct holder of the abstraction (or any other level in between)?
- from what i know, in pd the 'home' of any instance of any patch or
abstraction has been and still is the _file location_ of the patch or abstraction, no matter where you instantiate them from (menu->open or [path/to/file]). the 'relative to parent' approach would break this.
- if i am not totally mistaken, pd doesn't distinguish between
abstraction and patch, but they are both very much treated the same. i think, this is a strength of pd, because it makes patching very scalable: what once was a patch, can be used as an abstraction without any changes. pd 0.40 breaks this by ignoring [declare -path] inside abstractions, which is a bad and hopefully temporary solution, imo. the 'relative to parent' approach would definitely introduce the distinction between abstractions and patches for almost no gain, imo.
if possible in whatever way, i really would like to push the consensus towards the 'relative to file location' approach, in the name of the church of consistency and also just for the finding a consensus' sake.
(sorry, if i am pushing to hard.. too much beer) roman
Telefonate ohne weitere Kosten vom PC zum PC: http://messenger.yahoo.de
Roman Haefeli wrote:
On Wed, 2008-07-30 at 10:01 +0200, Frank Barknecht wrote:
Hallo, Matt Barber hat gesagt: // Matt Barber wrote:
When 0.39 begins to wane (so [declare] can be used), ...
Careful here: [declare -path ...] is disabled inside of abstractions in Pd-0.41.
Right -- but [declare -path ...] is terribly useful for not having a patch's main directory cluttered with 100 abstractions, which was the main point... but since 0.39 is still widely in use I tend to avoid it unless it's for patches I know only I am going to be running. I guess it's okay to be conservative in some parts of life. =o)
OT -- out of curiosity, if it were to be enabled within an abstraction, would the -path be relative to the abstraction file or to the patch in which it's instantiated?
Relative paths are important if you use declare in the way you described here: to make shipping "bundles" easier by putting all used abstractions into a subdirectory and adding that subdir to the searchpath of the MAIN.pd. As you don't know where a user puts your project, you cannot use absolute paths.
Now inside of an abstraction I would discern two use cases: One is using declare just as in the MAIN patch to add further subdirectories for other helper abstractions. Here the only thing that makes sense IMO when using relative paths is to have them relative to the abstraction itself.
Another use case is the path for "resources" like soundfiles or sequencer patterns. The example Miller once mentioned on pd-dev was having a [soundfiler] in a sample player abstraction. Usually this abstraction would live somewhere far away from your soundfiles or your MAIN.pd.
[soundfiler] also looks for soundfiles in the pd search path, so users may feel inclined to manipulate the path to make the sampler abstraction find wav-files in a "snd"-directory somewhere without giving the full absolute path. Here it probably is more common to use paths relative to MAIN.pd instead of sampleplayer.pd so the MAIN-file's -path setting would reach into the child patch as well.
A drum synth abstraction may be different again: Maybe it has some default samples for kick and hihat in a subdirectory next to itself. Now you could manipulate the path to make it find these samples first. Again absolute paths don't work here, as your synth.pd may be installed anywhere.
As there is no consensus which of the two path alternatives should be used inside an abstraction, -path currently is disabled completely in an abstraction. (Btw. I don't know how an abstraction knows that it's used as an abstraction, not as MAIN.pd.)
i question, if someone is seriously voting for the 'relative to parent' approach, since it is full of troubles:
I think, I do. (if seriously is another question...) to me an abstraction is like an include function (like for example in php). let's say I create an abstraction that plays soundfiles and the filename can be given as an argument or messages, then I want to write [mysfplayer sounds/1.wav]. in this case, I want the file be relative to the "parent" patch, because the parent patch will be in my project folder, but the abstraction will be somewhere in an abstraction directory.
otoh, if there are files that mysfplayer is depending on, for example a bg image with playbuttons.. which are relative to the [mysfplayer] abstraction, then they would still be found under images/button.gif, because if the abstraction[mysfplayer] is in the search path, then the images folder should be in the path, too.
making all paths relative to abstraction is reduntant imho.
- it's not clear, what the parent is: is it the top-most patch or the
patch, that is the direct holder of the abstraction (or any other level in between)?
the topmost.
- from what i know, in pd the 'home' of any instance of any patch or
abstraction has been and still is the _file location_ of the patch or abstraction, no matter where you instantiate them from (menu->open or [path/to/file]). the 'relative to parent' approach would break this.
I don't think it would break it, how?
- if i am not totally mistaken, pd doesn't distinguish between
abstraction and patch, but they are both very much treated the same. i think, this is a strength of pd, because it makes patching very scalable: what once was a patch, can be used as an abstraction without any changes. pd 0.40 breaks this by ignoring [declare -path] inside abstractions, which is a bad and hopefully temporary solution, imo. the 'relative to parent' approach would definitely introduce the distinction between abstractions and patches for almost no gain, imo.
if possible in whatever way, i really would like to push the consensus towards the 'relative to file location' approach, in the name of the church of consistency and also just for the finding a consensus' sake.
At least for the example above (mysfplayer) I don't see how this should work with the "relative to file location" approach....
marius.
On Wed, 2008-07-30 at 21:09 -0400, marius schebella wrote:
Roman Haefeli wrote:
On Wed, 2008-07-30 at 10:01 +0200, Frank Barknecht wrote:
Hallo, Matt Barber hat gesagt: // Matt Barber wrote:
When 0.39 begins to wane (so [declare] can be used), ...
Careful here: [declare -path ...] is disabled inside of abstractions in Pd-0.41.
Right -- but [declare -path ...] is terribly useful for not having a patch's main directory cluttered with 100 abstractions, which was the main point... but since 0.39 is still widely in use I tend to avoid it unless it's for patches I know only I am going to be running. I guess it's okay to be conservative in some parts of life. =o)
OT -- out of curiosity, if it were to be enabled within an abstraction, would the -path be relative to the abstraction file or to the patch in which it's instantiated?
Relative paths are important if you use declare in the way you described here: to make shipping "bundles" easier by putting all used abstractions into a subdirectory and adding that subdir to the searchpath of the MAIN.pd. As you don't know where a user puts your project, you cannot use absolute paths.
Now inside of an abstraction I would discern two use cases: One is using declare just as in the MAIN patch to add further subdirectories for other helper abstractions. Here the only thing that makes sense IMO when using relative paths is to have them relative to the abstraction itself.
Another use case is the path for "resources" like soundfiles or sequencer patterns. The example Miller once mentioned on pd-dev was having a [soundfiler] in a sample player abstraction. Usually this abstraction would live somewhere far away from your soundfiles or your MAIN.pd.
[soundfiler] also looks for soundfiles in the pd search path, so users may feel inclined to manipulate the path to make the sampler abstraction find wav-files in a "snd"-directory somewhere without giving the full absolute path. Here it probably is more common to use paths relative to MAIN.pd instead of sampleplayer.pd so the MAIN-file's -path setting would reach into the child patch as well.
A drum synth abstraction may be different again: Maybe it has some default samples for kick and hihat in a subdirectory next to itself. Now you could manipulate the path to make it find these samples first. Again absolute paths don't work here, as your synth.pd may be installed anywhere.
As there is no consensus which of the two path alternatives should be used inside an abstraction, -path currently is disabled completely in an abstraction. (Btw. I don't know how an abstraction knows that it's used as an abstraction, not as MAIN.pd.)
i question, if someone is seriously voting for the 'relative to parent' approach, since it is full of troubles:
I think, I do. (if seriously is another question...) to me an abstraction is like an include function (like for example in php). let's say I create an abstraction that plays soundfiles and the filename can be given as an argument or messages, then I want to write [mysfplayer sounds/1.wav]. in this case, I want the file be relative to the "parent" patch, because the parent patch will be in my project folder, but the abstraction will be somewhere in an abstraction directory.
otoh, if there are files that mysfplayer is depending on, for example a bg image with playbuttons.. which are relative to the [mysfplayer] abstraction, then they would still be found under images/button.gif, because if the abstraction[mysfplayer] is in the search path, then the images folder should be in the path, too.
making all paths relative to abstraction is reduntant imho.
- it's not clear, what the parent is: is it the top-most patch or the
patch, that is the direct holder of the abstraction (or any other level in between)?
the topmost.
i still think this approach is much more troublesome. why should a [declare -path] inside an abstraction expand the home of the topmost parent patch while keeping the scope within the abstraction itself? this is two levels of nesting too much and completely not necessary. this is hard to follow and wouldn't pd life make easier at all. if you want a path relative to the topmost parent patch, then this path should also be declared in the topmost parent patch. if you want to use global pathes for the whole project, then those pathes should only be declared in the topmost patch.
- from what i know, in pd the 'home' of any instance of any patch or
abstraction has been and still is the _file location_ of the patch or abstraction, no matter where you instantiate them from (menu->open or [path/to/file]). the 'relative to parent' approach would break this.
I don't think it would break it, how?
for the 'relative to parent' approach you would need to set the abstractions home to the topmost parent's one, so that it expands pathes correctly. 'relative to file location' is not only very easy to understand, it also doesn't change how pathes are handled now.
- if i am not totally mistaken, pd doesn't distinguish between
abstraction and patch, but they are both very much treated the same. i think, this is a strength of pd, because it makes patching very scalable: what once was a patch, can be used as an abstraction without any changes. pd 0.40 breaks this by ignoring [declare -path] inside abstractions, which is a bad and hopefully temporary solution, imo. the 'relative to parent' approach would definitely introduce the distinction between abstractions and patches for almost no gain, imo.
if possible in whatever way, i really would like to push the consensus towards the 'relative to file location' approach, in the name of the church of consistency and also just for the finding a consensus' sake.
At least for the example above (mysfplayer) I don't see how this should work with the "relative to file location" approach....
put the declare in the topmost patch. either you have the sample folder in a project specific place, then it also needs to be declared in the project's main patch, or it is always at the same place relative to 'mysfplayer', then it could be declared inside 'mysfplayer'.
ok, i see, that by proposing this i make some assumptions, that probably aren't obvious (i make those, because it seems to me the only obvious way to do it):
- pathes are inherited from the (any) parent
- inherited pathes are already expanded pathes, which means: '-path ../yoyo' from the parent doesn't expand the abstractions home with '../yoyo', but instead the abstraction uses '/path/to/parent/../yoyo', which is inherited form the parent.
yo.. let us try to find use cases, that _cannot_ be covered by this approach. if we don't find them, then let us please find the consensus (aka. going for the 'relative to file location' route).
roman
___________________________________________________________ Der frühe Vogel fängt den Wurm. Hier gelangen Sie zum neuen Yahoo! Mail: http://mail.yahoo.de
-
Damian Stewart -
Frank Barknecht -
Hans-Christoph Steiner -
IOhannes m zmoelnig -
Luke Iannini -
marius schebella -
Matt Barber -
Roman Haefeli