On Jun 14, 2018, at 10:21 AM, pd-list-request@lists.iem.at wrote:

From: Roman Haefeli <reduzent@gmail.com>
To: Pd-List <pd-list@lists.iem.at>
Subject: Re: [PD] standard paths for externals
Message-ID: <0b50bb13c24e850d65000e980526f8234a138f61.camel@gmail.com>
Content-Type: text/plain; charset="utf-8"

Hey Dan

Thanks for trying to explain.

On Thu, 2018-06-14 at 01:18 +0200, Dan Wilcox wrote:
There seems to be a bit of FUD in this "discussion."

Maybe things aren't that bad as they were presented. For my part, I
lost track of who favors what kind of design.

Some things from my perspective:

1. Pd's library/path mechanism is largely oriented toward *local*
libs first and *global* ones later, ie. download externals to
individual project (sub)folders.

Pd already used to work like that for quite a while, no?

I believe the centralized loading was added later on in Pd-extended. 

I don't understand what you mean by that. Pd-extended already pre-
loaded a subset of the installed externals (do you mean this?), which -
imho - was a bad design choice. It meant there was one person deciding
what comes pre-loaded and what not.

Yes and continuing that mechanism by installing things into more "standard places" is perhaps less ideal, even if it is easy, ie. ~/Library/Pd.

If we follow the "programming language model", this might be similar
to installing a Python egg locally or globally. I don't see how
supporting both modes of working is a problem.

Me neither.

2. IOhannes convinced me, over time, that the "stdpath" loading
mechanism used by the "extra" folder is problematic as it adds too
many (sub) search paths by default and it becomes harder to
tell/control what's being loaded.

Forgive my ignorance, but I don't understand. Does Pd now look into
extra's subfolders? Currently I still have to actively load stuff with
[declare -stdlib] or [declare -stdpath]. 

It always has, that's one of the "standard paths" aka those that [decalre's -stdpath] uses. These are different from the "user search paths" which are those  specified in the Path dialog and/or via [declare -path]. Similarly, [declare -stdlib] search for lib names *only* in the "standard paths" while [decalre -lib] looks only in supplied "user search paths". This is where great confusion comes and arguments over what is a path and how things should/can be searched.

Our utltimate solution is to move to dis-favor the separation and simply have [declare -path] and [dlecare -lib] search both the "standard paths" and the "user search paths", first locally, then the "suer paths", and finally the more global "standard paths".

The "better way" is for people to specify those externals paths
directly, either with the user search paths and/or declare.
There is much less ambiguity as to what's going on at the loss of
"just put things in this place" ease.

So paths added by the users will be searched by deken? So  instead of
specifying paths to the library root - as we currently do it - adding a
path in the future means adding a searchpath that is container for many
libraries? And deken will find libraries inside the newly added
searchpath root? 

By deken, do you mean [declare]?

Deken is unrelated to path searching, all it does is download things to a place, either the ~/Documents/Pd/externals directory or some other places specified by you.




3. As suggested by Alexandre, I wrote a PR to help address how
declare search when using -path & -lib by having them search along
user search paths as well: https://github.com/pure-data/pure-
data/pull/205. I feel this is a much closer method of using declare
to how other programming languages handle such things, ie. Lua's
require.

OK, I hopefully understand now. Am I right in thinking that declare's
-stdlib, -stdpath flags are considered superfluous because -lib and    
-path will scan all searchpaths in a local -> global order, including
the parent patch's folder?

That's the general idea. As I mention before, it's hard to tell which to use since people tend to put things in different places and, as you mention in a previous email, where is the "right" place to put things. I think the "right place" should be wherever you want as long as [declare] can find it *without* requiring the distinction between -path and -stdpath.


4. The Pd Documents path (aka ~/Documents/Pd) is simply a helper
location for beginners. It is only a regular user search path
added by default (when desired) and does not search subfolders. 

Now you challenge my understanding of the term searchpath again. I
thought the new idea is that you will be able to load libraries with
[declare] from user-added searchpaths? What's the point of creating
that folder then? 

The folder is created to give you a place where externals can be downloaded. Previously, deken either installed externals inside the app bundle's /extra folder on macOS (hidden "standard path") or asked you *each time* where to put things. This simply automates where things can go for beginners, removing a pain point and follows the Arduio/Processing model.

The next step is the downloaded external needs to be added to the user search paths for things to work. This is why deken asks you if you want to add the path after you download the external and it's sort of a kludge until something like the [declare -path] fix can be integrated. It makes things "work" for now for beginners (and people missing extended) with two clicks. I know IOhannes is not a fan but, again, it's a stopgap.

Nobody has to use it and it can be disabled at any time.

My beef with it is that - the way it is presented to me -  it suggests
to do things that create confusion later. 

As a new user:

"Do you want to create ~/Documents/Pd"

Yes

"Do you want to add it to the searchpaths?"

Yes

When downloading stuff with deken: "Do you want to install 'mylib' to
~/Documents/Pd?"

Yes

Now, I'm going to load mylib with [declare -{std}path mylib] in my
patch which fails.

What am I missing here?

The fact that this won't work with [declare] yet... unless you install to a "standard path", ie /extra etc. Again, we're trying to move away form this model.


5. Yes, the macOS hiding the ~/Library folder is problematic for many
users as they don't know how to show it and are afraid to modify
things within it. I've recently taught classes to beginners that are
just learn gin the concept of a "file system" as they grew up mainly
using "mobile phones"

I don't see how deken requires users to have a notion about a
filesystem. Ideally, I'd use deken to tell _what_ I want to install.
Ideally, I'd tell my patch through [declare] _what_ to load. All the
_where_ stuff shouldn't be a question neither at installing nor loading
time. The _where_ stuff should only matter when the user tweaks their
environment: "I'd rather put my 4.5 TB of Pd externals on my external
drive ". 

Yeah, and that's the whole point. You should be free to put things wherever and you are.

The issue is that [declare -lib] currently doesn't look beyond local paths.

The whole file system thing is about creating a basic place for beginners who can later on learn more about the path mechanisms and be bale to handle it themselves. We're trying to find a way for people who "just want to make stuff" to do so without running into the (from what I can tell) immense pain of moving from Pd-extended to suddenly having to manage externals themselves. Please don't discount this as supporting beginners should be the health part of *any* open source project, otherwise it's just a bunch of "grey beards" complaining about fewer and fewer "newbs."

--------
Dan Wilcox
@danomatika
danomatika.com
robotcowboy.com