I started working on the pd autotools build mainly because I had already used autotools before and it seemed like the easiest way to get Pd to build on macOS again. So far, we've moved forward with it in general, but at this point I wonder if a re-examination makes sense.
Some points:
./configure && make are great for users -> The more people who can easily build Pd, the more help we can get for all sorts of things. I think we've been seeing that the last 6 months or so.
Built-in support for gettext processing, standard install targets, etc allow for translations and package management (Debian)
*Maintaining* the configure.ac & Makefile.am files makes maintaining the build system relatively easy, if you understand it (see next point).
*Developing* the autotools build system is a PITA. There are a lot of details and basic understanding required in order to both make sense and use it effectively. Once everything works, it's *great* (point 1), but sometimes getting there takes a while. Also, fewer people are willing to help with it because of the overhead.
Pd is in some ways both a simple and a complex project to build. Simple in that C is easy to build anywhere and TCL just needs things in the right place. It's complex in that Pd builds and runs on a variety of platforms, each with it's own peculiarities.
The original makefile build system works relatively well for the simple cases while the autotools build system works for the complexities (while requiring complexity). The makefiles are straightforward and easy to understand at a loss in flexibility while ./configure et al are black boxes that support lots of configuration options and hopefully work most of the time.
When I did the overhaul last year, I went through and added documentation to the various Makefile.ams and the configure.ac to help in understanding things, but I think that it's still down to IOhannes and I to do any sort of work on the setup as it is. I'm not sure if that is healthy in the long run.
I'm thinking now about about different approaches to simplify what is there:
non-recursive automake: a single large Makefile.am without the various files in subfolders. There are a number of advantages to this including build speed, putting things in one place, etc.
hybrid autoconf/makefile: If we want, we can use regular makefile rules in automake files or our own makefiles instead of automake and simply use the configuration variables for autoconf. This would allow for exposing more of the core build information in a readable way at the expense of perhaps higher maintenance.
The autotools are definitely useful but I admit can be a bit of a beast (but this guide helps a great deal: https://autotools.io/index.html https://autotools.io/index.html). OTOH I would argue that *any* build system can be a beast, especially for a cross-platform project. In either case, I don't think we want to end up with a "non-understandable black box mess" and I'm thinking of ways to make things easy to work with for both Pd users *and* developers.
Dan Wilcox @danomatika http://twitter.com/danomatika danomatika.com http://danomatika.com/ robotcowboy.com http://robotcowboy.com/
some comments from someone who very recently got into autotools. my personal problem was that I thought just because automake generates makefiles, it would be similar to make syntax. after I understood it's a totally different system and followed some tutorials, I think it's not so hard. there are some quirks like getting the right flags for libtool to produce proper Windows DLLs, tricking libtool to link with g++ for ASIO etc. but most other platform specific knowledge will be the same for any build system I guess. after all, Pd changes in a slow and healthy pace, so there aren't many modifications to be done to the build system once it is set up.
the one thing I noticed is build speed: a complete rebuild (including externals) on my personal MinGW build system takes ~40 seconds but with autotools it takes a whopping 4 minutes and 30 seconds!
- hybrid autoconf/makefile: If we want, we can use regular makefile rules in automake files or our own makefiles instead of automake
and simply use the configuration variables for autoconf.
that could also be a good solution. ./configure is definitly great to have!
anyway, just some thoughts, I don't have any real opinion. I'm willing to work on whatever build system we finally get.
Gesendet: Sonntag, 03. Dezember 2017 um 03:58 Uhr Von: "Dan Wilcox" danomatika@gmail.com An: pd-list pd-list@iem.at Cc: "Miller Puckette" msp@ucsd.edu Betreff: [PD] autotools
I started working on the pd autotools build mainly because I had already used autotools before and it seemed like the easiest way to get Pd to build on macOS again. So far, we've moved forward with it in general, but at this point I wonder if a re-examination makes sense. Some points:
- ./configure && make are great for users -> The more people who can easily build Pd, the more help we can get for all sorts of things. I think we've been seeing that the last 6 months or so.
2. Built-in support for gettext processing, standard install targets, etc allow for translations and package management (Debian) 3. *Maintaining* the configure.ac & Makefile.am files makes maintaining the build system relatively easy, if you understand it (see next point). 4. *Developing* the autotools build system is a PITA. There are a lot of details and basic understanding required in order to both make sense and use it effectively. Once everything works, it's *great* (point 1), but sometimes getting there takes a while. Also, fewer people are willing to help with it because of the overhead. Pd is in some ways both a simple and a complex project to build. Simple in that C is easy to build anywhere and TCL just needs things in the right place. It's complex in that Pd builds and runs on a variety of platforms, each with it's own peculiarities. The original makefile build system works relatively well for the simple cases while the autotools build system works for the complexities (while requiring complexity). The makefiles are straightforward and easy to understand at a loss in flexibility while ./configure et al are black boxes that support lots of configuration options and hopefully work most of the time. When I did the overhaul last year, I went through and added documentation to the various Makefile.ams and the configure.ac to help in understanding things, but I think that it's still down to IOhannes and I to do any sort of work on the setup as it is. I'm not sure if that is healthy in the long run. I'm thinking now about about different approaches to simplify what is there:
- non-recursive automake: a single large Makefile.am without the various files in subfolders. There are a number of advantages to this including build speed, putting things in one place, etc.
2. hybrid autoconf/makefile: If we want, we can use regular makefile rules in automake files or our own makefiles instead of automake and simply use the configuration variables for autoconf. This would allow for exposing more of the core build information in a readable way at the expense of perhaps higher maintenance. The autotools are definitely useful but I admit can be a bit of a beast (but this guide helps a great deal: https://autotools.io/index.html). OTOH I would argue that *any* build system can be a beast, especially for a cross-platform project. In either case, I don't think we want to end up with a "non-understandable black box mess" and I'm thinking of ways to make things easy to work with for both Pd users *and* developers.
Dan Wilcox @danomatika[http://twitter.com/danomatika] danomatika.com[http://danomatika.com] robotcowboy.com[http://robotcowboy.com] _______________________________________________ Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list%5Bhttps://lists.puredata.info/l...]
On Dec 3, 2017, at 11:39 AM, Christof Ressi christof.ressi@gmx.at wrote:
but most other platform specific knowledge will be the same for any build system I guess. after all, Pd changes in a slow and healthy pace, so there aren't many modifications to be done to the build system once it is set up.
That is definitely true. Automake puts together the makefile rules for you but of course it still works in the same way in the end.
I've also found this guide to be pretty helpful in the getting into the basics: https://autotools.io/index.html https://autotools.io/index.html
the one thing I noticed is build speed: a complete rebuild (including externals) on my personal MinGW build system takes ~40 seconds but with autotools it takes a whopping 4 minutes and 30 seconds!
Yeah and that's largely due to the current layout being recursive, ie. multiple makefiles in multiple directories. This is one of the reasons there have been some projects that rework things into a single main makefile as it lowers the amount of directory walking and intermediate steps ie. non-recursive make: https://autotools.io/automake/nonrecursive.html https://autotools.io/automake/nonrecursive.html
To some degree, configure will always be slow for all sorts of reasons, but there are definitely techniques to slow down the make process. Another advantage to non-recursive make is that doing a parallel build would work without timing uncertainties between different makefiles.
Dan Wilcox @danomatika http://twitter.com/danomatika danomatika.com http://danomatika.com/ robotcowboy.com http://robotcowboy.com/
Yeah and that's largely due to the current layout being recursive,
hmmm.. I don't think that's the reason. currently, there aren't that many Makefiles: /src, /portaudio, /portmidi, /asio, /extra and for each external. aside from that, my build system is also recursive (calling make on pd-lib-builder makefiles for the externals). what's really slow is compiling the individual source files. while a typical .c file in /src takes ~1 second on my build system, it takes significantly longer with libtool. this is easily noticeable when watching the console output. I think it's rather an overhead introduced by libtool but I might be wrong. did you notice that on other platforms?
Gesendet: Sonntag, 03. Dezember 2017 um 12:55 Uhr Von: "Dan Wilcox" danomatika@gmail.com An: "Christof Ressi" christof.ressi@gmx.at Cc: pd-list pd-list@iem.at, "Miller Puckette" msp@ucsd.edu Betreff: Re: [PD] autotools
On Dec 3, 2017, at 11:39 AM, Christof Ressi <christof.ressi@gmx.at[mailto:christof.ressi@gmx.at]> wrote: but most other platform specific knowledge will be the same for any build system I guess. after all, Pd changes in a slow and healthy pace, so there aren't many modifications to be done to the build system once it is set up. That is definitely true. Automake puts together the makefile rules for you but of course it still works in the same way in the end. I've also found this guide to be pretty helpful in the getting into the basics: https://autotools.io/index.html%5Bhttps://autotools.io/index.html] the one thing I noticed is build speed: a complete rebuild (including externals) on my personal MinGW build system takes ~40 seconds but with autotools it takes a whopping 4 minutes and 30 seconds! Yeah and that's largely due to the current layout being recursive, ie. multiple makefiles in multiple directories. This is one of the reasons there have been some projects that rework things into a single main makefile as it lowers the amount of directory walking and intermediate steps ie. non-recursive make: https://autotools.io/automake/nonrecursive.html%5Bhttps://autotools.io/autom...] To some degree, configure will always be slow for all sorts of reasons, but there are definitely techniques to slow down the make process. Another advantage to non-recursive make is that doing a parallel build would work without timing uncertainties between different makefiles.
Dan Wilcox @danomatika[http://twitter.com/danomatika] danomatika.com[http://danomatika.com] robotcowboy.com[http://robotcowboy.com]
Yeah, I'm looking into not using libtool actually, partially for this reason. We are using it with port audio and port midi, but it's not really required for things which are only statically linked in the project. The dll we could leave as something we do more by hand as we (you!) have worked out the set of steps needed.
it's recommended to only use libtool for when it's actually necessary and this is really for building cross-platform shared libraries. It will be great for libpd, for instance, but I'm thinking now we don't really need it for desktop Pd.
On Dec 3, 2017, at 1:39 PM, Christof Ressi christof.ressi@gmx.at wrote:
Yeah and that's largely due to the current layout being recursive,
hmmm.. I don't think that's the reason. currently, there aren't that many Makefiles: /src, /portaudio, /portmidi, /asio, /extra and for each external. aside from that, my build system is also recursive (calling make on pd-lib-builder makefiles for the externals). what's really slow is compiling the individual source files. while a typical .c file in /src takes ~1 second on my build system, it takes significantly longer with libtool. this is easily noticeable when watching the console output. I think it's rather an overhead introduced by libtool but I might be wrong. did you notice that on other platforms?
Dan Wilcox @danomatika http://twitter.com/danomatika danomatika.com http://danomatika.com/ robotcowboy.com http://robotcowboy.com/
maybe for MinGW, automake could just pass the right variables to makefile.mingw? I could also rework the MinGW makefiles for the externals so that they don't depend on pd-lib-builder.
Gesendet: Sonntag, 03. Dezember 2017 um 13:42 Uhr Von: "Dan Wilcox" danomatika@gmail.com An: "Christof Ressi" christof.ressi@gmx.at Cc: pd-list pd-list@iem.at Betreff: Re: [PD] autotools
Yeah, I'm looking into not using libtool actually, partially for this reason. We are using it with port audio and port midi, but it's not really required for things which are only statically linked in the project. The dll we could leave as something we do more by hand as we (you!) have worked out the set of steps needed. it's recommended to only use libtool for when it's actually necessary and this is really for building cross-platform shared libraries. It will be great for libpd, for instance, but I'm thinking now we don't really need it for desktop Pd.
On Dec 3, 2017, at 1:39 PM, Christof Ressi <christof.ressi@gmx.at[mailto:christof.ressi@gmx.at]> wrote:
Yeah and that's largely due to the current layout being recursive, hmmm.. I don't think that's the reason. currently, there aren't that many Makefiles: /src, /portaudio, /portmidi, /asio, /extra and for each external. aside from that, my build system is also recursive (calling make on pd-lib-builder makefiles for the externals). what's really slow is compiling the individual source files. while a typical .c file in /src takes ~1 second on my build system, it takes significantly longer with libtool. this is easily noticeable when watching the console output. I think it's rather an overhead introduced by libtool but I might be wrong. did you notice that on other platforms?
Dan Wilcox @danomatika[http://twitter.com/danomatika] danomatika.com[http://danomatika.com] robotcowboy.com[http://robotcowboy.com]
-
Christof Ressi -
Dan Wilcox