On Apr 21, 2009, at 3:09 PM, Claude Heiland-Allen wrote:

Hi Hans, all,

Hans-Christoph Steiner wrote:

...

I don't know if any of the active pure:dyne packagers are on this list, but I thought I'd try. From what I have seen, it looks like pure:dyne is doing a much better job of making .deb packages than Pd-extended. I would be great to 'officially' merge efforts so that we don't duplicate work.

Sure, we strive for high quality packages. Regarding duplication of work, the pure:dyne wiki [1] has some packaging tips, the main thing is to get a good source.deb that can then be built for any Debian- based distro. We have a pbuilder machine ready to compile packages, if anyone wants to help broaden the range of pure data related packages in pure:dyne (and eventually Debian main) you are more than welcome to join in the effort.

[1] http://code.goto10.org/projects/puredyne/wiki/DevLounge#SoftwarePackaging

Regarding pd-extended, pure:dyne is based on pd-vanilla, which we've found to be quite stable, while pd-extended seems to have more experimental patches that don't always work out.

The key here is to make sure that the library packages can work with separate versions of pd. Something like 'puredata' and 'pd-extended' which both provide 'pd' but can coexist. That means the libraries should probably be installed into a standardized shared location, so maybe instead of /usr/lib/pd, use /usr/lib/pd-externals, which would match ~/pd-externals/ and /usr/local/lib/pd-externals for user- installed stuff.

Then you can keep transparent boxed, bold fonted Pd in pure:dyne ;-) and others can choose a different look. I think basically, Pd-vanilla is barely maintained on Windows and Mac OS X, and is maintained on to work well on a stripped down GNU/Linux setup. Pd-extended aims to feel native on the most common platforms: GNOME, Mac OS X, and Windows. I think we've done a lot of good work along those lines. AFAICT, pure:dyne aims to be a stripped down GNU/Linux install, so it makes sense to use Pd-vanilla.

Also pd-extended's policy of splitting every library into tiny pieces solves one problem but causes others, so I think it was slightly premature to do the splitting until the other issues are fixed. pure:dyne policy as far as there is one is to build the way the author(s) intended, resulting in a mix of libraries and single- object externals - but our job as packagers (in the .deb sense) isn't to save the pd universe with some grand architecture, but simply to make packages available for users :)

When you guys encounter problems with it, I would greatly appreciate feedback so that those problems can be addressed. The library format came about to address rear world problems. For example, using [mylib/ myobj] is currently the _only_ way to ensure that you get the object that you expect, regardless of whether its a .pd_linux or .pd. And [mylib/myobj] works with _any_ version of Pd back to 0.30 or before. This is quite important when making libraries that don't break with each new release of pure:dyne or whatever. The introduction of [pow~] to Pd-vanilla 0.42-4 demonstrated this point.

With abstractions the situation is worse. If you make your abstraction called [threshold] and include it in the same folder as your project. Then you use a patch that uses smlib's [threshold] and close it. Reopen your patch and [threshold] will be the smlib one, _not_ your threshold.pd. Or say Miller adds [threshold] to Pd- vanilla, same story, except there is no way to ever load your threshold.pd, unless you stick into a folder and call it [myfolder/ threshold].

"Use it like it is because its like that" seems like surrender to me. I think we can solve the hard problems if we try. Two steps forward, one step back is an inevitable part of the process. When you link libraries into one file, then you cannot address any of these name conflicts. I am happy to settle on using only what works now, but since pure:dyne is only Linux, [declare -path] and [declare -lib] work now, for example.

A big part of these problems with Pd-extended comes from having so many libraries loaded by default. I think that none of the libraries should be loaded by default, I am guessing that's how pure:dyne does it.

...

Then we can make some really nice, proper packages and get them into Debian.

Makes sense, that's one of the key aims of pure:dyne too.

...

DebConf 2010 is in NYC, and I'll be helping to run it, so it would be great to have all this in Debian by then. Plus, it seems that Guenter has less time for the Debian packages, so things like the pd-externals package has lapsed.

That sounds like a good target. The pd-externals package is long obsolete (last updated 2004 iirc).

...

It seems to me that the next step would be to get the pure:dyne stuff into pure-data SVN, then make any tweaks to it to make it work with Pd-extended too.

We're more than happy to be good packagers and report bugs upstream, with any build patches required, but ideally there would be more upstream releases of known-quality code - currently it is still a bit hit-and-miss for some externals/libraries to know if there are unfinished pieces of code that will be fixed the next day or two, which acts as a disincentive for packagers to put the effort in to package from the svn.

Yes, this is a definitely something to think about. These days, I am thinking more and more that we should make it easy for people to package and release libraries themselves, and make it really easy to install libraries. That's the first step. Once we have a whole bunch of Pd externals in SVN that are set up with clean .deb building code, there will be lots of examples for people to draw from when packaging their own libraries.

Personally I'm not in favour of keeping debian/ folders in the same svn as the upstream source code, as they have rather orthogonal purposes.

If the aim is to make official debian packages, I think it makes sense to maintain the debian/control, etc files in the main Pd repo: pure- data SVN.

.hc

----------------------------------------------------------------------------

There is no way to peace, peace is the way. -A.J. Muste

On Apr 24, 2009, at 6:40 AM, Claude Heiland-Allen wrote:

Hi,

Hans-Christoph Steiner wrote:

...

On Apr 21, 2009, at 3:09 PM, Claude Heiland-Allen wrote:

...

Hi Hans, all,

Hans-Christoph Steiner wrote:

[snip]

...

The key here is to make sure that the library packages can work with separate versions of pd. Something like 'puredata' and 'pd- extended' which both provide 'pd' but can coexist.

Sure, that's possible with Debian packaging.

...

That means the libraries should probably be installed into a standardized shared location, so maybe instead of /usr/lib/pd, use / usr/lib/pd-externals, which would match ~/pd-externals/ and /usr/ local/lib/pd-externals for user-installed stuff.

WTF?? This is exactly the kind of pointless disruptive change that I was arguing strongly against here:

Could you illustrate the disruption? ~/pd-externals/ and /usr/local/ lib/pd-externals (and their equivalent for each platform) was included starting in Pd-extended 0.40.3 and it works very well to have a standard location for user-installed files. Having a standard location means we can have clear documentation as well as other benefits. Ideally, all versions of Pd would use the same locations to avoid disruptions and confusion, so I submitted a patch to Miller for this.

How then do you propose to support installing externals for multiple 'pd' packages? The 'extra' folder is really tied to a given version of Pd. Its one thing to complain about the 'pd-externals' idea, but its just a complaint unless you propose a solution, or at least describe the problems with it. I think that most packagers actually make many such changes to fit with Debian Policy. This is a good thing IMHO, it is one of the big strengths of Debian.

For example, a typical change that would also change the standard Pd paths is installing the HTML docs into /usr/share/doc/puredata instead of /usr/lib/pd/doc/1.manual. I think a proper Debian package of Pd should do this as well. I think that many Debian packagers would argue that no docs should be in /usr/lib/pd. Plus following Debian Policy, it should be /usr/lib/puredata anyway, since the package name is used as the identifier as well.

I think we can easily manage such changes without disruptions with some careful attention.

...

...

our job as packagers (in the .deb sense) isn't to save the pd universe with some grand architecture, but simply to make packages available for users :)

[snip]

...

...

Also pd-extended's policy of splitting every library into tiny pieces solves one problem but causes others, so I think it was slightly premature to do the splitting until the other issues are fixed.

[snip]

...

When you guys encounter problems with it, I would greatly appreciate feedback so that those problems can be addressed.

[>~] etc

Yes, I think though IOhannes' valiant efforts, we have learned that we need to have a special case to deal with objects that include the characters < > / \ | * ? " : (these are not supported all filesystems). For those, they can be loaded from a single file. I think this could be supported in a nextgen libdir based on IOhannes' idea for having a binary shared library that is loaded when the library is loaded. Then we can have a single folder that can have a shared binary library for the external, the possibility to load objects with < > / \ | * ? ", abstractions included in the same library, and the help patches also included in the same library.

As for characters that can be on the filesystem, but not in C function names, I think we should just have a generic setup() function for those, like Max/MSP does (tho there is called main()).

I tried to sum up the years of discussion and what I think is the beginnings of a consensus for a solution in my PdCon paper. I would love feedback on it.

http://at.or.at/hans/Let%27s_Make_Libraries_-_PdCon3.pdf

[snip]

...

With abstractions the situation is worse. If you make your abstraction called [threshold] and include it in the same folder as your project. Then you use a patch that uses smlib's [threshold] and close it. Reopen your patch and [threshold] will be the smlib one, _not_ your threshold.pd. Or say Miller adds [threshold] to Pd- vanilla, same story, except there is no way to ever load your threshold.pd, unless you stick into a folder and call it [myfolder/ threshold].

Sure, but that's nothing to do with Debian packaging.

...

"Use it like it is because its like that" seems like surrender to me.

We're talking about packaging for Debian, not saving the world in one jump. When the issues are improved (by the upstream authors), they make a new upstream release, then the packager can update the Debian package.

The decision as a pd externals packager is which library format to use for externals which have more than one feasible format. That's where this comes into the packaging discussion.

...

When you link libraries into one file, then you cannot address any of these name conflicts.

True, but [>~]

...

A big part of these problems with Pd-extended comes from having so many libraries loaded by default. I think that none of the libraries should be loaded by default, I am guessing that's how pure:dyne does it.

The live distribution has a ~/.pdrc that loads most things.

FYI: Miller and others consider the .pdrc deprecated, so the package should use .pdsettings, IMHO. But that's not a big deal. The Pd GUI only uses .pdsettings.

...

Yes, this is a definitely something to think about. These days, I am thinking more and more that we should make it easy for people to package and release libraries themselves, and make it really easy to install libraries. That's the first step.

Yes. We (as packagers) can only package what is there.

...

Once we have a whole bunch of Pd externals in SVN that are set up with clean .deb building code, there will be lots of examples for people to draw from when packaging their own libraries.

You can use "apt-get source" to get some examples from the pure:dyne repositories already.

...

...

Personally I'm not in favour of keeping debian/ folders in the same svn as the upstream source code, as they have rather orthogonal purposes.

If the aim is to make official debian packages, I think it makes sense to maintain the debian/control, etc files in the main Pd repo: pure-data SVN.

Why? The Pure-data repository is for Pure-data things, and Debian has its own infrastructure for Debian things.

upstream authors (write externals) | | upstream source repository V upstream maintainer (tidy up externals for release) | | upstream source release V ------ upstream responsibility ends here debian maintainer (packages release for debian) ------ debian responsibility begins here | | debianized source release V debian build system / repository (builds for debian users) | | debian packages V user "aptitude install" (gets to enjoy the externals)

Here's how I see it, there are three general cases:

- Many of us are authors and packagers, so why separate the debian part?

- there are many authors who are not packagers but use Debian. I imagine that we could convince such authors to maintain the packaging code once its setup. debian/control and debian/rules are not hard to read and edit once they are setup, so I think we could get many people to maintain their own packages if the debian code was in the same place.

- Then for other situations I imagine it would often make more sense to maintain the packaging code in a different repo.

.hc

Thanks,

Claude

http://claudiusmaximus.goto10.org http://puredyne.goto10.org http://goto10.org

----------------------------------------------------------------------------

¡El pueblo unido jamás será vencido!