win32 build farm 'msys.exe' (WAS: symlinks render pdstring unbuildable on MinGW/Windows)
moin Hans,
On 2009-04-18 06:57:53, Hans-Christoph Steiner hans@eds.org appears to have written:
The bad news is that it seems that my bad diagnosis led you on a wide goose chase thru the pain of Windows development.
It may or may not have been a bad diagnosis (although it certainly was a pain ;-) -- I still haven't verified the former, since I don't use (or understand) all of the auto-build scripts. I'm still kinda hoping IOhannes might make some headway there...
Apparently, string2any and friends are still not getting built. In fact all of the 'moocow' is empty on Windows. Here's the bug report:
https://sourceforge.net/tracker/?func=detail&aid=2138593&group_id=55...
I noticed this too, and am currently trying to figure out exactly why. Actually, I didn't really expect any of my externals to build on windows currently, since I was missing test clauses for win32 in ax_pd_external.m4. I've added the test clauses now, and at least win32 on the build farm machine is getting properly recognized, but I'm getting a whole mess of other (stupid) errors which are preventing manual compilation:
'msys.bat' doesn't run on the autobuild machine from the cygwin shell I get when I ssh in: is this (easily) fixable? I can run "/cygdrive/c/msys/1.0/bin/sh" manually, but it's not really useable as an interactive shell. So far I've been explicitly calling "$MSYS_SH ./configure" to test, for MSYS_SH=/cygdrive/c/msys/1.0/bin/sh.exe
I've tried manually prepending "/cygdrive/c/msys/1.0/bin:/cygdrive/c/MinGW/bin" to PATH, and that gets me all the mingw programs, but their compiled-in paths (e.g. include paths for gcc) are foobared, since cygwin's pseudo-filesystem doesn't jive with the msys one. In particular, I'm getting:
pddev@windowsxp-i386$ sh ./configure \ --with-pd-include="/home/pd/auto-build/pd-extended/pd/src" ... checking for string.h... no configure: WARNING: could not find standard C headers -- things may get ugly
... similar problems with linker paths. I've hacked this for now by setting CPPFLAGS and LDFLAGS, and the updated version of pdstring seems to build ok.
Part of the problem might be MinGW's very old version of autotools: automake 1.7 and libtool 1.4, which is a big bummer.
This shouldn't be a problem, since I've been storing autoconf-generated stuff in svn anyways, and calling AM_MAINTAINER_MODE to ensure that it doesn't get implicitly rebuilt. See the thread "[PD-dev] moocow: svn and compilation issues" for more details.
In a related question, should the auto-build process for the win32 build-farm machine be trying to build everything from the LIB_TARGETS variable in externals/Makefile? If so, then I must be being pretty dense, because I can't see where my builds are getting called (I see install calls, but no configuration or compilation), much less where they might be failing...
marmosets, Bryan
On Mar 7, 2009, at 5:14 PM, Bryan Jurish wrote:
moin Hans,
Short version: should be fixed now. package-local links to the common code are now no longer tracked by SVN, but kludged into place by an "./svn-prepare.sh" (symlink ;-) in each package directory, alternately by the DIR.autogen_stamp target in extended/Makefile.
More comments: see below.
On 2009-03-05 23:44:58, Hans-Christoph Steiner hans@eds.org appears to have written:
On Mar 5, 2009, at 5:22 PM, Bryan Jurish wrote:
On 2009-03-05 21:33:36, Hans-Christoph Steiner hans@eds.org appears to have written:
On Mar 4, 2009, at 4:23 PM, Bryan Jurish wrote:
On 2009-03-04 05:37:40, Hans-Christoph Steiner hans@eds.org appears to have written: > I was just trying to build string2any and friends on Windows for a > student, but the symlinks used in moocow are throwing a huge wrench > in > the process. > They show up at .lnk files, and are not links at all. > That's because cygwin translates symlinks into Windows shortcuts, > aka .lnk.
[snip]
the rsync "-L" flag ("--copy-links") works for me here, even with a preceding "-a" ("--archive")... does that not work on cygwin? The only times I've ever had problems with "-L" were symlink cycles (./dir -> .), which I certainly am not inserting into the repository.
Sorry, I had no intention to insult or demean, I guess I was just terse.
No worries; the whole mess really arises from my not having the time to dig around in the automake internals (bad programmer, no biscuit): I'm sure there's a way to get the common code into its own automake+autoconf package, but I haven't figured it out yet. Ideally, I'd like to have an family of automake "_PDEXTERNALS" targets (analagous to "_PROGRAMS", "_LIBRARIES", "_DATA", etc.), but that's not happening yet; hence the intermediate solution "../common", which goes pretty far towards eliminating the headaches necessary to roll up a new external package or add new functionality to an existing package.
The bad news is that its not that simple. I added "--no-l --copy-links" to cygwin rsync and it still doesn't work.
Curioser and curioser. I just tried an rsync from linux (with symlinks) to cygwin with "-a --no-l --copy-links" here, and I got copies of the symlinks rather than windoof shortcuts. Can you point me at the (script containining the) full rsync call so I can test that here?
There is nothing you have to do here, I just thought it'd be nice to have those externals included in Windows. Here are the logs, it seems it doesn't find the compiler properly:
http://bxmc.poly.edu/pdlab/moocow_log.txt http://bxmc.poly.edu/pdlab/config.log
403 forbidden
not really important though, since the AC_CONFIG_LINKS() symlinks aren't the problem.
What I don't understand is why this code needs such a complicated build system? As far as I can tell, it is mostly pretty standard C code.
If you're asking that now, it's probably a good thing I didn't delegate external building to libtool ;-)
It's (mostly) not the *code* which needs automake, it's *me* who wants it. Having 'distcheck' and 'dist' targets generated auto-magically is really outrageously spiffy. And with the shared code, building a new external in an existing package is as easy as setting a couple of make variables:
pdexterns_PROGRAMS += myexternal myexternal_SOURCES = myexternal.c mycommon.c mycommon.h
... and automake takes care of the rest (make, make install, make uninstall, ...). Creating a new external package using this system is pretty easy too: see the SVN directory externals/moocow/hello for a complete working tutorial, or its README here: http://pure-data.svn.sourceforge.net/viewvc/pure-data/trunk/externals/moocow...
The ./configure options are there for debugging, support for multiple installations of Pd on the same machine, as well as support for pd-extended (e.g. --enable-object-externals).
autotools support for use environment variables (CFLAGS, CPPFLAGS, etc.) is really handy too, e.g. for messing with optimization & debugging flags. I wish every external build system would support these.
That said, in some cases the code actually *does* require some platform-dependent initialization in ./configure. [locale] for instance checks for definitions of all of the "LC_*" variables it supports, [flite] needs to know where to find the libflite include files, as well as how to link to libflite: this is exactly the kind of thing autoconf was made to handle, and which it does quite elegantly.
I find that in the long run, simple Makefiles are the least work overall. To each his own, I suppose, or maybe I'm missing something.
If I were maintaining only 1 or 2 external packages (or a single global build system ;-), I think I might tend to agree. As it is, I think the autotools beat copy+paste Makefiles (which I still use a lot, e.g. for my LaTeX documents, which rarely need to be synchronized or updated once the paper has been written) hands down. Are automake+autoconf+autoheader overkill for my externals? Sure they are: I think this is demonstrated pretty well by the fact that [sprinkler] and [pdstring] built for years in what became flatspace/ with 2-liner C files to the tune of:
#define PACKAGE_VERISON "cvs" #include "../../moocow/sprinkler/sprinkler.c"
Did flatspace builds get me 'make dist'? Nope. Did they get me 'make distcheck'? No way. Could I configure them for target systems to which I myself have no access simply by querying the availability of the relevant C libraries and includes? Uh... yeah, right... enough; I love externals/build rsp. flatspace and everything it's done for me, and I really don't want to start a flame war... I hope the windoof builds work now (though I am still curious about that apparent rsync bug)...
marmosets, Bryan
--Bryan Jurish "There is *always* one more bug." jurish@ling.uni-potsdam.de -Lubarsky's Law of Cybernetic Entomology
I spent 33 years and four months in active military service and during that period I spent most of my time as a high class muscle man for Big Business, for Wall Street and the bankers. - General Smedley Butler
-
Bryan Jurish