Today I built the sources from the pd-double autobuild on OSX 10.5. In order to force double precision I added --enable-double-precision to the PD_CONFIGURE_FLAGS in the Makefile for darwin_app. This gives a double precision core, but unfortunately the externals are still single precision.
Impatiently, I've set PD_FLOAT_PRECISION in m_pd.h to 64 here. Two weeks after the announcement of double precision Pd-extended, I'm eager to finally know what it looks like.
Well? It refuses to build. Or more precisely: some functions in some files in some libs can't be compiled, mostly because of 'incompatible pointer type'. Nothing so serious, but these libs have to be omitted for the moment until they're repaired, otherwise pd-double-extended can not even be built for testing. The culprits are:
creb cw_binaural extra smlib vbap
About bsaylor and unauthorized I don't know because there were unsatisfied dependencies for them on my system. I can repair 'extra' tomorrow and commit to pd-double.git.
By the way you can use the help file for [cyclone/mean] to check whether the cyclone lib (and probably the whole of pd-extended) is built with the same precision as the core, because this class is 'by it's nature' double-precision-ready. If it gives garbage output, the build is wrong, not the C code. Hopefully there's many more double-precision-ready classes in the external libs, that's what we'll find out shortly.
Katja
On Thu, 2011-10-13 at 01:28 +0200, katja wrote:
Today I built the sources from the pd-double autobuild on OSX 10.5. In order to force double precision I added --enable-double-precision to the PD_CONFIGURE_FLAGS in the Makefile for darwin_app. This gives a double precision core, but unfortunately the externals are still single precision.
Impatiently, I've set PD_FLOAT_PRECISION in m_pd.h to 64 here. Two weeks after the announcement of double precision Pd-extended, I'm eager to finally know what it looks like.
Sounds like a good dev setup. The stuff I committed is more about trying to automatically set things up for people building themselves. It definitely an open question how to handle this. Like your situation, what's the easiest way to force double on 10.5 or other 32-bit platform? Another way would be to add -DPD_FLOAT_PRECISION=64 to the CFLAGS in packages/darwin_app/Makefile.
Well? It refuses to build. Or more precisely: some functions in some files in some libs can't be compiled, mostly because of 'incompatible pointer type'. Nothing so serious, but these libs have to be omitted for the moment until they're repaired, otherwise pd-double-extended can not even be built for testing. The culprits are:
creb cw_binaural extra smlib vbap
If you want SVN commit access, check the puredata.info dev wiki for the instructions.
About bsaylor and unauthorized I don't know because there were unsatisfied dependencies for them on my system. I can repair 'extra' tomorrow and commit to pd-double.git.
One thing: there is the stuff in pd/extra, then there is a copy of that stuff with an alternate build system in externals/extra. They should be in sync code-wise. But just be sure to only commit to pd/extra in the pd-double.git, not externals/extra in the pure-data SVN.
By the way you can use the help file for [cyclone/mean] to check whether the cyclone lib (and probably the whole of pd-extended) is built with the same precision as the core, because this class is 'by it's nature' double-precision-ready. If it gives garbage output, the build is wrong, not the C code. Hopefully there's many more double-precision-ready classes in the external libs, that's what we'll find out shortly.
I think most should be ready to go, as long as they used t_float and t_sample well. Mostly, if they used t_float for both t_float and t_sample, which is pretty common, my guess is that will work too.
.hc
On Thu, Oct 13, 2011 at 11:53 PM, Hans-Christoph Steiner hans@at.or.at wrote:
what's the easiest way to force double on 10.5 or other 32-bit platform? Another way would be to add -DPD_FLOAT_PRECISION=64 to the CFLAGS in packages/darwin_app/Makefile.
Strangely, -DPD_FLOAT_PRECISION=64 does not work at all, a pure single precision build seems to result. Looking at the command line output, all the other options are implemented when building the externals. Well not all of them, -DDEBUG_SOUNDFILE is also missing in the output. Normally the -D defines are shown in the output if I remember well. (The core and the extra's by the way, have -O2 and -g but not the explicit vector/sse options). Anyway, the only method of building a double precision Pd on 10.5 is now by just commenting out the conditional compilation stuff and set PD_FLOAT_PRECISION hardcoded to 64.
If you want SVN commit access, check the puredata.info dev wiki for the instructions.
When I've found out how to best handle all the different sources, I'll introduce myself to the list as a wannabe pd-dev...
One thing: there is the stuff in pd/extra, then there is a copy of that stuff with an alternate build system in externals/extra. They should be in sync code-wise. But just be sure to only commit to pd/extra in the pd-double.git, not externals/extra in the pure-data SVN.
Indeed I stumbled upon that extra extra when trying to find the puzzling cause of errors. They are not in sync, externals/extra has the old code, that's why it fails to build. The extra's in pd-double.git and are double-ready.
I think most should be ready to go, as long as they used t_float and t_sample well. Mostly, if they used t_float for both t_float and t_sample, which is pretty common, my guess is that will work too.
The underlying primitive type is what counts for the compiler. The trickiest obstacles for double precision are type punnings. I suspect cyclone will be full of it, since it's modeled on Max/Msp and has the same style of routines as Pd. For the rest, I have no clue of what to expect.
Katja
On Oct 13, 2011, at 8:37 PM, katja wrote:
On Thu, Oct 13, 2011 at 11:53 PM, Hans-Christoph Steiner <hans@at.or.at
wrote:
what's the easiest way to force double on 10.5 or other 32-bit platform? Another way would be to add -DPD_FLOAT_PRECISION=64 to the CFLAGS in packages/darwin_app/Makefile.
Strangely, -DPD_FLOAT_PRECISION=64 does not work at all, a pure single precision build seems to result. Looking at the command line output, all the other options are implemented when building the externals. Well not all of them, -DDEBUG_SOUNDFILE is also missing in the output. Normally the -D defines are shown in the output if I remember well. (The core and the extra's by the way, have -O2 and -g but not the explicit vector/sse options). Anyway, the only method of building a double precision Pd on 10.5 is now by just commenting out the conditional compilation stuff and set PD_FLOAT_PRECISION hardcoded to 64.
This sounds like a good dev setup, so you can force the setup that you want. It'll be a tricky situation to handle down the road, I don't have a clear idea of how it should be done, but if things are working for you, I'm happy to leave it for later.
If you want SVN commit access, check the puredata.info dev wiki for the instructions.
When I've found out how to best handle all the different sources, I'll introduce myself to the list as a wannabe pd-dev...
One thing: there is the stuff in pd/extra, then there is a copy of that stuff with an alternate build system in externals/extra. They should be in sync code-wise. But just be sure to only commit to pd/extra in the pd-double.git, not externals/extra in the pure-data SVN.
Indeed I stumbled upon that extra extra when trying to find the puzzling cause of errors. They are not in sync, externals/extra has the old code, that's why it fails to build. The extra's in pd-double.git and are double-ready.
Hmm, are you sure? I just re-copied the files from pure-data.git to externals/extra and svn tells me there is no difference. The do have different build systems, but the .pd, .c, and .h files I think are all the same.
I think most should be ready to go, as long as they used t_float and t_sample well. Mostly, if they used t_float for both t_float and t_sample, which is pretty common, my guess is that will work too.
The underlying primitive type is what counts for the compiler. The trickiest obstacles for double precision are type punnings. I suspect cyclone will be full of it, since it's modeled on Max/Msp and has the same style of routines as Pd. For the rest, I have no clue of what to expect.
I've added a daily type-punning log that just grabs the gcc warning from the build logs and consolidates them, its posted here every day: http://autobuild.puredata.info/auto-build/2011-10-13/logs/type-punning.log
.hc
----------------------------------------------------------------------------
All mankind is of one author, and is one volume; when one man dies, one chapter is not torn out of the book, but translated into a better language; and every chapter must be so translated.... -John Donne
On Fri, Oct 14, 2011 at 5:27 AM, Hans-Christoph Steiner hans@at.or.at wrote:
On Oct 13, 2011, at 8:37 PM, katja wrote:
Indeed I stumbled upon that extra extra when trying to find the puzzling cause of errors. They are not in sync, externals/extra has the old code, that's why it fails to build. The extra's in pd-double.git and are double-ready.
Hmm, are you sure? I just re-copied the files from pure-data.git to externals/extra and svn tells me there is no difference. The do have different build systems, but the .pd, .c, and .h files I think are all the same.
Yes externals/extra are the same as the extra's in pure-data.git, with all the floats still there, instead of t_float. If externals/extra can be synched with the extra's in pd-double.git, we're fine.
I've added a daily type-punning log that just grabs the gcc warning from the build logs and consolidates them, its posted here every day: http://autobuild.puredata.info/auto-build/2011-10-13/logs/type-punning.log
Very useful. It's quite a list, but be sure it's not complete!
When looking at the logs, I also found that today's macosx106-x86_64 pd-double autobuild failed. That was to be expected. Because of the precision-to-bitness coupling in m_pd.h, it was now building the externals in double precision for the first time (the earlier builds all have single precision externals). gcc exited with an error when compiling creb, just like I've seen it on my own computer.
The point is, I have no clue how to create a dev setup for even the simplest commit. I was mimicking the nightly build on my computer just to see where problems are, which libs do not compile etc. But it's not a useful route for development. On the other hand, in a local pd-svn, externals can only be built against pd-double when old pd is overwritten with pd-double.git. But then it's no longer a working copy of pure-data SVN, and does not allow for update or commit. And from within pd-double git dir, I could not get an external lib to be built with the symlink method as described on puredata.info.
Pd-double-extended has a very weird status: partly it's a branch or fork, partly it's a form of bugfixing. So you can commit untested changes which are safe, like changing float into t_float, and then in the nightly build check if it worked out well with pd-double. But some modifications, for example the replacement of Hoeldrich style routines and other type punning stuff, must be tested thoroughly on functionality, robustness and performance before committing. By lack of a formal branch where all the double-ready code is unified, this work can only be done locally, with a dev set up yet to figure out. In any case, such substantial rewrites can be submitted to the patch tracker, so others can test them before they're committed. It won't concern too many cases. For example, in the core, less than ten classes on a total of ~200 needed a fundamental redefinition to make them double-ready. (Even so I've spent over a month on that, due to unfamiliarity with pd's core code).
Is this an idea? - simple changes like float > t_float: untested, via SVN - substantial function redefinitions: via patch tracker
In this way, the fact that one can't build and test for pd-double in pd-svn is less of a problem. Then we can think about a dev set up from where you'd only produce patch files, and not commit to SVN. That should give more options of how to organize it. At the same time, it gives some guarantee that redefinitions with possible performance- or functionality- consequences are not committed too fast.
Katja
On Friday, October 14, 2011 2:48 PM, "katja" katjavetter@gmail.com wrote:
On Fri, Oct 14, 2011 at 5:27 AM, Hans-Christoph Steiner hans@at.or.at wrote:
On Oct 13, 2011, at 8:37 PM, katja wrote:
Indeed I stumbled upon that extra extra when trying to find the puzzling cause of errors. They are not in sync, externals/extra has the old code, that's why it fails to build. The extra's in pd-double.git and are double-ready.
Hmm, are you sure? I just re-copied the files from pure-data.git to externals/extra and svn tells me there is no difference. The do have different build systems, but the .pd, .c, and .h files I think are all the same.
Yes externals/extra are the same as the extra's in pure-data.git, with all the floats still there, instead of t_float. If externals/extra can be synched with the extra's in pd-double.git, we're fine.
I've added a daily type-punning log that just grabs the gcc warning from the build logs and consolidates them, its posted here every day: http://autobuild.puredata.info/auto-build/2011-10-13/logs/type-punning.log
Very useful. It's quite a list, but be sure it's not complete!
When looking at the logs, I also found that today's macosx106-x86_64 pd-double autobuild failed. That was to be expected. Because of the precision-to-bitness coupling in m_pd.h, it was now building the externals in double precision for the first time (the earlier builds all have single precision externals). gcc exited with an error when compiling creb, just like I've seen it on my own computer.
The point is, I have no clue how to create a dev setup for even the simplest commit. I was mimicking the nightly build on my computer just to see where problems are, which libs do not compile etc. But it's not a useful route for development. On the other hand, in a local pd-svn, externals can only be built against pd-double when old pd is overwritten with pd-double.git. But then it's no longer a working copy of pure-data SVN, and does not allow for update or commit. And from within pd-double git dir, I could not get an external lib to be built with the symlink method as described on puredata.info.
Just replacing the pd/ folder with the pd-double.git as pd/ does not affect the rest of the code in the SVN tree. It is all still fully functional SVN. That's a big difference between how git and SVN work. It is one advantage of SVN, in that it allows you to create whack setups like this and still have it functional ;). This is how I've been working on Pd-extended for years.
That said, you can still treat most libraries as standalone units, certainly any lib that is built on the Library Template. So if you prefer, you can work that way. You will probably need to set the PD_INCLUDE variable then to point to your pd-double.git/src, something like:
make PD_INCLUDE=-I~/code/pd-double.git/src
You can also build libraries individually using the Pd-extended build system:
cd pd-double/externals/ make DESTDIR=/tmp creb_clean creb creb_install
Pd-double-extended has a very weird status: partly it's a branch or fork, partly it's a form of bugfixing. So you can commit untested changes which are safe, like changing float into t_float, and then in the nightly build check if it worked out well with pd-double. But some modifications, for example the replacement of Hoeldrich style routines and other type punning stuff, must be tested thoroughly on functionality, robustness and performance before committing. By lack of a formal branch where all the double-ready code is unified, this work can only be done locally, with a dev set up yet to figure out. In any case, such substantial rewrites can be submitted to the patch tracker, so others can test them before they're committed. It won't concern too many cases. For example, in the core, less than ten classes on a total of ~200 needed a fundamental redefinition to make them double-ready. (Even so I've spent over a month on that, due to unfamiliarity with pd's core code).
Is this an idea?
- simple changes like float > t_float: untested, via SVN
- substantial function redefinitions: via patch tracker
In this way, the fact that one can't build and test for pd-double in pd-svn is less of a problem. Then we can think about a dev set up from where you'd only produce patch files, and not commit to SVN. That should give more options of how to organize it. At the same time, it gives some guarantee that redefinitions with possible performance- or functionality- consequences are not committed too fast.
That works for me, as long as you ask if its ok before you commit to a certain library. I don't think you need to ask per file, per-library is probably fine-grained enough. To start with, while I'm not the original author, I am the maintainer of many libs in pure-data SVN. Here are some you can commit to directly now for the pd-double project (other features should be discussed before hand):
bassemu~ boids cxc creb ext13 freeverb~ ggee hcs markex maxlib mjlib moonlib motex pdogg plugin~ sigpack smlib windowing
It sounds like you have a good process already for working on this: code, test, commit, test. It doesn't need to be perfect before committing, as long as you test before committing and follow up on any issues. This is also a great opportunity to write some automated test patches. I've been working on the infrastructure for running an automated nightly test suite. The first part is working, it is the 'load every help' log that gets posted on the auto-build site each day. The next step is to start writing Pd patches that test Pd itself. Then I can start plugging those in.
I'm thinking it'll automatically run any patch called *-regressiontest.pd, then check if that patch outputs 'SUCCESS' within a timeout period.
.hc
-
Hans-Christoph Steiner -
katja