On Jul 1, 2010, at 12:33 PM, dmotd wrote:
hi hans,
okay, i still have no real idea what you are talking about here? i get the makefile template, but how exactly does this relate to a buildsystem, the examples that use the template are still called by the centralised makefile?
yes, check ext13 externals/Makefile for example.
thomas maintains his own buildsystem and as much as its apparently stumped a bunch of people in the past, it is designed for cross platform AND max/msp. to redesign an entire build system to fit a template, for sake of maintaining some design rules seems a little over the top, that's why i decided to simply wrap it. besides, the flext layer has enough complexity to require a decent autoconf first, and if recent threads are correct, there's no libdir based autoconf template.
A libdir autoconf template would be awesome to have though. Building C or C++ objects for Pd isn't hard. What about flext makes things so complicated? In the process of making that template Makefile, I realized that autotools is really rarely needed for externals. Only really when there are a lot of library dependencies.
the thing that has me confused is this notion of packaging? if pd-X is going to disolve into a system of separated libs (libdirs?) called by a virtual package 'pd-X', and thus 'debianize' the entire system, then i support that - however there must be some wrapper script that does the job that pd-X currently does? i understand that the template is designed to make that job simpler, by making the build assumptions the responsibility of each lib and creating a libdir path, but how does this actually spawn packages?
what i like about pd-X is that it unifies a lot of extras and provides a neat wrapper for the build, so i'm assuming that your direction will be providing scripts to wrap certain build environments (osx/win/fed/deb/ubu/etc). have you got examples of how this next generation of build scripts will look with the makefile-template, or will the current centralised system slowly morph into the desired endpoint? i am a little concerned that a 'one size fits all' may not fit all...
The core idea is to avoid one-size-fits-all by decentralizing the libraries. If libraries are really easy to make, distribute and install, then there isn't a strong need to have all the libraries included in Pd-extended itself. People can maintain their libraries entirely separately, removing the unavoidable roadblocks that happen when things get centralized.
For Pd-extended 0.43, I think we should be removing a bunch of libraries that are not really used or maintained and make them separately distributed. You can see some idea of the plans here:
http://puredata.info/dev/NextRelease
That said, there is value to having a default set of libraries that you know will always be included with Pd-extended. I think to get to that point, libraries need to be very stable both in terms of their interface/features and their bugginess. Think of python libraries, or ruby, etc.
.hc
anyway, without some guidelines, i'm not sure i will ever understand what you have in mind, and what is acceptable practice. i'm just wanting to contribute something that's been missing from the pd-x object pool, without getting too caught up in some fundamental change of practice.
best, dmotd
Hans-Christoph Steiner wrote:
The idea is to have all libraries have their own standalone build systems that do not rely on the packages/Makefile.buildlayout stuff. That's the idea with the Makefile/library template.
I think the best plan for flext would be to make a version of the template Makefile that works for C++/flext. Then that Makefile can easily be used for Debian/Ubuntu/RPM/etc. packaging.
http://puredata.info/docs/developer/MakefileTemplate
I could see including the flext library itself in Pd-extended, that is if it is not changing much. But the externals itself should start out separately distributed libdirs IMHO.
.hc
On Jun 28, 2010, at 9:50 PM, dmotd wrote:
hi hans,
the work i've done slots straight into the existing externals/Makefile structure and currently builds fine on linux, it uses thomas' build system and until a better autoconf alternative is created, this is by far the easiest (and has been working quite well for years). as such this a multi-class single lib system.
i have followed the libdir rules, with help-files abstraction (and in py/ext case scriptets) in each libs own dir, and other stuff in examples. the actual flext headers and libs get installed in their own $prefix/include $prefix/lib.
my intention is to get the flext externals into the nightly autobuid. i'm not sure what you mean by 'distribuited as stand-alone', if this is a reference to your plan to split supported pd-extended libs into individual packages, then i'll need to know more about what scripts you have at your disposal and your release schedule for this new packaging system.
i've attached a diff of my working makefile, which will need to be tested against other platforms and linux distributions. currently the following libs build and install correctly: absattr, pool, py, xsample (i am waitinng on some fixes up stream for clk and vasp)
sorry for not emailing earlier, i've had little time to work on this stuff and just ran with the momentum with the small time i have had.
cheers, dmotd
Hans-Christoph Steiner wrote:
Hey, I just saw you are working flext stuff, lots of people will appreciate that. I don't know your plan, but I wanted to save you some effort. At this point, I think that the flext stuff should be distributed as standalone libraries, rather than integrated with Pd- extended. That makes it easier to package too, for things like archlinux, Debian, Fedora, etc.
Also, if you are starting in on packaging things for archlinux, check out the externals/template. If you can make a template archlinux packaging file for that, then it'll be really easy to make packages for all of the libraries that use that template:
http://puredata.info/docs/developer/MakefileTemplate
.hc
News is what people want to keep hidden and everything else is publicity. - Bill Moyers
<flext.diff>
I hate it when they say, "He gave his life for his country." Nobody gives their life for anything. We steal the lives of these kids. - Admiral Gene LeRocque
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
----------------------------------------------------------------------------
Information wants to be free. -Stewart Brand
Hans-Christoph Steiner wrote:
yes, check ext13 externals/Makefile for example.
yes yes, i know the makefile template well, what i'm interested in is more a proposal/example for how different environments will package these templated libs, dpkg/rpm/zip/dmg/etc, what wrapper scripts will be needed? how will modularized libdirs get installed in environments like mac and windows?
thomas maintains his own buildsystem and as much as its apparently stumped a bunch of people in the past, it is designed for cross platform AND max/msp. to redesign an entire build system to fit a template, for sake of maintaining some design rules seems a little over the top, that's why i decided to simply wrap it. besides, the flext layer has enough complexity to require a decent autoconf first, and if recent threads are correct, there's no libdir based autoconf template.
A libdir autoconf template would be awesome to have though. Building C or C++ objects for Pd isn't hard. What about flext makes things so complicated? In the process of making that template Makefile, I realized that autotools is really rarely needed for externals. Only really when there are a lot of library dependencies.
flext is a library dependecy!
flext itself is a programming interface to create a standard api between pd + max/msp, the library and its headers need to be installed in a known path and thus becomes a dependency to any library using flext. additionally it has optional simd sse/altivec optimizations, and can be extended further with sndobj and stk (synthesis tookit).
the current flext buildsys compiles the libs reliant on flext from a single build script, using definitions set in each libs src folder (package.txt), thus each lib is aware of the environment settings that flext was compiled with and some libs (eg. py/ext) require the user to manipulate an additional configuration file to define other system settings.
i concede that it may be possible to make some libs easily conform to the makefile template, but then the pd template is rather useless for building the same lib for max/msp, which if i recall is a popular reason for using flext in the first place.
The core idea is to avoid one-size-fits-all by decentralizing the libraries. If libraries are really easy to make, distribute and install, then there isn't a strong need to have all the libraries included in Pd-extended itself. People can maintain their libraries entirely separately, removing the unavoidable roadblocks that happen when things get centralized.
hang on! you are decentralizing building (back to) a makefile in each directory, and the makefile is based on a template, which according to 'you shouldn't need to edit anything below here' appears to be a 'one-size-fits-all' approach. this is good and all, and i totally undestand the idea and necessity, but the problem i see occuring is when people do begin to edit below the line to accomodate a slightly more demanding build technique. what happens when the template itself needs updating - a sed script to find and replace every instance, hoping that someone hasn't made a minor alteration?
currently its quite simple to make those sort of changes very swiftly, as the build directives are centralized and one change effects the whole pack. i'm just a little worried that the modular template may become more difficult to maintain in the long term.
For Pd-extended 0.43, I think we should be removing a bunch of libraries that are not really used or maintained and make them separately distributed. You can see some idea of the plans here:
http://puredata.info/dev/NextRelease
That said, there is value to having a default set of libraries that you know will always be included with Pd-extended. I think to get to that point, libraries need to be very stable both in terms of their interface/features and their bugginess. Think of python libraries, or ruby, etc.
essentially what you wish to include with pd-X is your choice and you set the flavour of that release. thats one of many benefits of decentralized packaging, there's great scope in building unique virtual targets for more niche audiences - ie, a pd for matrix maths, a pd for hardware interfaces, a pd for audio analysis and mapping, a pd for visual artists and vjs etc etc etc..
but code maturity is a difficult call, and i think an interesting working bee would be itemizing and orphaning problem objects and libs, finding duplicate behaviour and seeing what niche demand there is for legacy buggy code. that way regardless of flavour the objects being distributed will yeild the best possible results for the user.
-dmotd
A libdir autoconf template would be awesome to have though. Building C or C++ objects for Pd isn't hard. What about flext makes things so complicated? In the process of making that template Makefile, I realized that autotools is really rarely needed for externals. Only really when there are a lot of library dependencies.
flext is a library dependecy!
flext itself is a programming interface to create a standard api between pd + max/msp, the library and its headers need to be installed in a known path and thus becomes a dependency to any library using flext. additionally it has optional simd sse/altivec optimizations, and can be extended further with sndobj and stk (synthesis tookit).
i guess for a start it would be ok to leave out SIMD as well as sndobj and STK, although it should definitely be includable later on. Compilation of the flext library is really straightforward. The few preprocessor defs should not really require autoconf.
Also, when flext availability within pd-ext is given, most of the flext-based externals can be built with fixed path settings for the flext headers and libraries (not requiring autoconf either). py/pyext and others are notable exceptions, though.
the current flext buildsys compiles the libs reliant on flext from a single build script, using definitions set in each libs src folder (package.txt), thus each lib is aware of the environment settings that flext was compiled with and some libs (eg. py/ext) require the user to manipulate an additional configuration file to define other system settings.
i concede that it may be possible to make some libs easily conform to the makefile template, but then the pd template is rather useless for building the same lib for max/msp, which if i recall is a popular reason for using flext in the first place.
Maybe it's ok to ignore Max for all autobuild-related questions, because the trusty flext build system can still be used for that.
gr~~~
On Jul 2, 2010, at 2:06 AM, dmotd wrote:
Hans-Christoph Steiner wrote:
yes, check ext13 externals/Makefile for example.
yes yes, i know the makefile template well, what i'm interested in is more a proposal/example for how different environments will package these templated libs, dpkg/rpm/zip/dmg/etc, what wrapper scripts will be needed?
Check out the template library mentioned in the MakefileTemplate page, it is a working debian package. I'm currently working with someone to make it do RPMs too. Also, all of the libs mentioned in that page are also debian packages.
how will modularized libdirs get installed in environments like mac and windows?
Take your pick:
a) run 'make' then zip up the folder for people to copy into ~/Library/ Pd, ~/pd-externals, etc. b) run 'make' then copy the library folder in ~/Library/Pd, ~/pd- externals, etc. c) run 'make install' and it'll install into ~/Library/Pd, ~/pd- externals, etc.
thomas maintains his own buildsystem and as much as its apparently stumped a bunch of people in the past, it is designed for cross platform AND max/msp. to redesign an entire build system to fit a template, for sake of maintaining some design rules seems a little over the top, that's why i decided to simply wrap it. besides, the flext layer has enough complexity to require a decent autoconf first, and if recent threads are correct, there's no libdir based autoconf template.
A libdir autoconf template would be awesome to have though. Building C or C++ objects for Pd isn't hard. What about flext makes things so complicated? In the process of making that template Makefile, I realized that autotools is really rarely needed for externals. Only really when there are a lot of library dependencies.
flext is a library dependecy!
flext itself is a programming interface to create a standard api between pd + max/msp, the library and its headers need to be installed in a known path and thus becomes a dependency to any library using flext. additionally it has optional simd sse/altivec optimizations, and can be extended further with sndobj and stk (synthesis tookit).
the current flext buildsys compiles the libs reliant on flext from a single build script, using definitions set in each libs src folder (package.txt), thus each lib is aware of the environment settings that flext was compiled with and some libs (eg. py/ext) require the user to manipulate an additional configuration file to define other system settings.
i concede that it may be possible to make some libs easily conform to the makefile template, but then the pd template is rather useless for building the same lib for max/msp, which if i recall is a popular reason for using flext in the first place.
Ah, more complicated than I thought. The other idea is to just make the flext build system create libdirs using its equivalent of 'make install'. Then if it respects $DESTDIR and $prefix, it'll work with the Pd-extended build system too.
The core idea is to avoid one-size-fits-all by decentralizing the libraries. If libraries are really easy to make, distribute and install, then there isn't a strong need to have all the libraries included in Pd-extended itself. People can maintain their libraries entirely separately, removing the unavoidable roadblocks that happen when things get centralized.
hang on! you are decentralizing building (back to) a makefile in each directory, and the makefile is based on a template, which according to 'you shouldn't need to edit anything below here' appears to be a 'one-size-fits-all' approach. this is good and all, and i totally undestand the idea and necessity, but the problem i see occuring is when people do begin to edit below the line to accomodate a slightly more demanding build technique. what happens when the template itself needs updating - a sed script to find and replace every instance, hoping that someone hasn't made a minor alteration?
currently its quite simple to make those sort of changes very swiftly, as the build directives are centralized and one change effects the whole pack. i'm just a little worried that the modular template may become more difficult to maintain in the long term.
Most Pd libraries do not have dependencies outside of Pd and the core C libs. It is for those cases that this Makefile template is for. The next step is to make an autotools template for more complicated situations.
Also, the Makefile is versioned, and its easy enough to ship 'diff - uw' patches to update the Makefile within a library. I've done that a few times already.
.hc
For Pd-extended 0.43, I think we should be removing a bunch of libraries that are not really used or maintained and make them separately distributed. You can see some idea of the plans here:
http://puredata.info/dev/NextRelease
That said, there is value to having a default set of libraries that you know will always be included with Pd-extended. I think to get to that point, libraries need to be very stable both in terms of their interface/features and their bugginess. Think of python libraries, or ruby, etc.
essentially what you wish to include with pd-X is your choice and you set the flavour of that release. thats one of many benefits of decentralized packaging, there's great scope in building unique virtual targets for more niche audiences - ie, a pd for matrix maths, a pd for hardware interfaces, a pd for audio analysis and mapping, a pd for visual artists and vjs etc etc etc..
but code maturity is a difficult call, and i think an interesting working bee would be itemizing and orphaning problem objects and libs, finding duplicate behaviour and seeing what niche demand there is for legacy buggy code. that way regardless of flavour the objects being distributed will yeild the best possible results for the user.
-dmotd
----------------------------------------------------------------------------
Terrorism is not an enemy. It cannot be defeated. It's a tactic. It's about as sensible to say we declare war on night attacks and expect we're going to win that war. We're not going to win the war on terrorism. - retired U.S. Army general, William Odom
-
dmotd -
Hans-Christoph Steiner -
Thomas Grill