On Sun, Oct 2, 2011 at 11:36 PM, Hans-Christoph Steiner hans@at.or.at wrote:
I think it makes sense to work off of pure-data.git rather than pd-extended.git since this is a patch targetted at getting into Miller's repo.
Right. Even then, we could add some external libs to work on, starting with zexy and cyclone for example. The question is how to load appropriate executables into local single and double precision test builds of vanilla Pd. A single preference file is shared by all vanilla Pd installations, therefore setting searchpaths via preferences is no option. Each Pd should only load from it's own 'extra' directory. With the extra's included in vanilla Pd, this works fine as far as I have seen. I tested bonk~ in single and double precision Pd simultaneously without symbol collision.
As for arch issues, I think Intel and ARM are the big ones to test these days. But PPC is fine too.
Expressed in number of 'users', ARM is probably the most popular target hardware for Pd at the moment. It should be easy to patch libpd with the same .patch files, or not? Some modified files are inexistent in libpd (s_audio_pa.c, vexpr.c etc). It's important to at least benchmark-test rewritten code on this hardware indeed, if we want to make a unified doube-precision-ready Pd happen.
Katja
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-10-03 14:28, katja wrote:
On Sun, Oct 2, 2011 at 11:36 PM, Hans-Christoph Steiner hans@at.or.at wrote:
I think it makes sense to work off of pure-data.git rather than pd-extended.git since this is a patch targetted at getting into Miller's repo.
Right. Even then, we could add some external libs to work on, starting with zexy and cyclone for example. The question is how to load appropriate executables into local single and double precision test builds of vanilla Pd. A single preference file is shared by all vanilla Pd installations, therefore setting searchpaths via preferences is no option. Each Pd should only load from it's own 'extra' directory. With the extra's included in vanilla Pd, this works
i'm not sure whether i really understand what you are saying here.
however, as long as a double-precision Pd looks into different paths than a single precision Pd, we won't have any problems. however, as soon as a double precision enabled Pd will find such an external in there, it will go kaboom.
possible solutions to the problem that come to my mind are: - - use a different filename suffix for double-precision externals. e.g. bonk~.pd_linux and bonk~.double.pd_linux pro: you can ship single and double precision files in a single zip file con: yet another special suffix
- - use a different setup function name for double precision, e.g. bonk_tilde_setup() vs bink_tilde_dsetup() pro: allows to have both single and double precision objects in a single binary con: needs extra code in each external (esp. if you want to exploit the 'pro' without resorting to C++-templates...) con: code need to be made aware for which width it is compiled
- - use a special exported function in the code, that indicates the width, for which the object was compiled [*] pro: backward compatible, con: needs extra code in each external (could probably done with a macro).
[*] something like: int pd_floattype_compatcheck(int runtimetype) { return runtimetype && COMPILETIMETYPE); } COMPILETIMETYPE would be defined in m_pd.h during load time, before Pd calls the _setup function, it would check for pd_floattype_compatcheck() and call it with runtimetype=COMPILETIMETYPE. if the compatcheck returns!=0, then the external is considered to be compatible with the currently used COMPILETIMETYPE and can thus be safely used. if 0 is returned, the external is incompatible and the setup function is not called at all (and hopefully the search for a compatible external is continued). if there is no pd_floattype_compatcheck() exported, then we really don't know whether we are compatible. i would suggest to assume compatibility, but make Pd issue a warning before it tries to load (and run) the object, so people would have a way to find out what caused the crash. alternatively, one could assume "single precision" if the compatcheck is missing, and refuse to load it in double precision mode.
fine as far as I have seen. I tested bonk~ in single and double precision Pd simultaneously without symbol collision.
which symbols are supposed to collide?
fgmasdr IOhannes
On Oct 3, 2011, at 9:12 AM, IOhannes m zmoelnig wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-10-03 14:28, katja wrote:
On Sun, Oct 2, 2011 at 11:36 PM, Hans-Christoph Steiner <hans@at.or.at
wrote:
I think it makes sense to work off of pure-data.git rather than pd-extended.git since this is a patch targetted at getting into Miller's repo.
Right. Even then, we could add some external libs to work on, starting with zexy and cyclone for example. The question is how to load appropriate executables into local single and double precision test builds of vanilla Pd. A single preference file is shared by all vanilla Pd installations, therefore setting searchpaths via preferences is no option. Each Pd should only load from it's own 'extra' directory. With the extra's included in vanilla Pd, this works
i'm not sure whether i really understand what you are saying here.
however, as long as a double-precision Pd looks into different paths than a single precision Pd, we won't have any problems. however, as soon as a double precision enabled Pd will find such an external in there, it will go kaboom.
possible solutions to the problem that come to my mind are:
- use a different filename suffix for double-precision externals.
e.g. bonk~.pd_linux and bonk~.double.pd_linux pro: you can ship single and double precision files in a single zip file con: yet another special suffix
- use a different setup function name for double precision, e.g.
bonk_tilde_setup() vs bink_tilde_dsetup() pro: allows to have both single and double precision objects in a single binary con: needs extra code in each external (esp. if you want to exploit the 'pro' without resorting to C++-templates...) con: code need to be made aware for which width it is compiled
- use a special exported function in the code, that indicates the
width, for which the object was compiled [*] pro: backward compatible, con: needs extra code in each external (could probably done with a macro).
[*] something like: int pd_floattype_compatcheck(int runtimetype) { return runtimetype && COMPILETIMETYPE); } COMPILETIMETYPE would be defined in m_pd.h during load time, before Pd calls the _setup function, it would check for pd_floattype_compatcheck() and call it with runtimetype=COMPILETIMETYPE. if the compatcheck returns!=0, then the external is considered to be compatible with the currently used COMPILETIMETYPE and can thus be safely used. if 0 is returned, the external is incompatible and the setup function is not called at all (and hopefully the search for a compatible external is continued). if there is no pd_floattype_compatcheck() exported, then we really don't know whether we are compatible. i would suggest to assume compatibility, but make Pd issue a warning before it tries to load (and run) the object, so people would have a way to find out what caused the crash. alternatively, one could assume "single precision" if the compatcheck is missing, and refuse to load it in double precision mode.
These all sound like good ideas to try. My only concern is that we might let the deployment issues distract from the issues at hand about getting it actually working first.
In terms of packaging, I can see having 64-bit distros run double- precision Pd for all packages, and 32-bit distros run single precision. That should cover the bulk of situations, the other situations can be covered by custom builds. Having all the 64-bit packages use double-precision Pd is of course going to happen after a while, once we have the bugs worked out. Here I can see an advantage of the monolithic Pd-extended package: its an easy, self-contained test bed.
I think this could also apply to the file extensions. The 64-bit ones, like .l_ia64, would assume double-precision, and 32-bit ones like .l_i386 would be single precision. I'm not yet sure what to do about .pd_linux. On Mac OS X, we'd still only need .pd_darwin since the fat binaries handle all the archs. Then the 64-bit code would be double-precision, and the 32-bit single.
.hc
fine as far as I have seen. I tested bonk~ in single and double precision Pd simultaneously without symbol collision.
which symbols are supposed to collide?
fgmasdr IOhannes
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk6JtKUACgkQkX2Xpv6ydvTZZgCgu8d045+iv0ju7wvgSTefJBXa ZfMAoIh2eVZ2uwgh7e5gkjU+itRw3IlT =vbHJ -----END PGP SIGNATURE-----
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
----------------------------------------------------------------------------
Terrorism is not an enemy. It cannot be defeated. It's a tactic. It's about as sensible to say we declare war on night attacks and expect we're going to win that war. We're not going to win the war on terrorism. - retired U.S. Army general, William Odom
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-10-03 16:31, Hans-Christoph Steiner wrote:
These all sound like good ideas to try. My only concern is that we might let the deployment issues distract from the issues at hand about getting it actually working first.
i'm definitely with you here. what is still missing in terms of "getting it actually working first"?
afaict, katja's patches do make pd itself double-precision ready (the patches might have to be reviewed as far as coding-style is concerned though)
otoh, i wouldn't start "porting" externals before we have a deployment strategy.
one important thing missing right now, is how to compile Pd in a given precision without having to edit m_pd.h technically i think that the define stuff and the like should go into a separate file "types.h" (probably "m_types.h") which is generated from m_types.h.in during configure time, and which is included by m_pd.h (which should remain non-generated) the question is, what miller would think of such a thing.
In terms of packaging, I can see having 64-bit distros run double-precision Pd for all packages, and 32-bit distros run single precision. That should cover the bulk of situations, the other situations can be covered by custom builds. Having all the 64-bit packages use double-precision Pd is of course going to happen after a while, once we have the bugs worked out. Here I can see an advantage of the monolithic Pd-extended package: its an easy, self-contained test bed.
definitely, the traditional Pd-extended will have an easier time here.
nevertheless, the advent of ~/pd-externals for the user has made things significantly more complicated in terms of "just works".
fgmasdr IOhannes
On Mon, Oct 3, 2011 at 10:19 AM, IOhannes m zmoelnig zmoelnig@iem.at wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-10-03 16:31, Hans-Christoph Steiner wrote:
These all sound like good ideas to try. My only concern is that we might let the deployment issues distract from the issues at hand about getting it actually working first.
i'm definitely with you here. what is still missing in terms of "getting it actually working first"?
afaict, katja's patches do make pd itself double-precision ready (the patches might have to be reviewed as far as coding-style is concerned though)
otoh, i wouldn't start "porting" externals before we have a deployment strategy.
one important thing missing right now, is how to compile Pd in a given precision without having to edit m_pd.h technically i think that the define stuff and the like should go into a separate file "types.h" (probably "m_types.h") which is generated from m_types.h.in during configure time, and which is included by m_pd.h (which should remain non-generated) the question is, what miller would think of such a thing.
Would you prefer to set the types at configure time through a file--or for example by adding a -DDOUBLE compiler flag? The affected locations of code defining the types could just use #ifdef DOUBLE
In either case, the configure option seems necessary. It still seems an open question how best to offer the double-precision types to externals developers.
In some cases, the setup() function allocates memory, which needs to be aware of the data type size. Otherwise, memory for signals is allocated through Pd's DSP graph generation routines, so that only changes to externals is to compile perform routines with the given data type.
Adding additional methods seems unnecessary--unless specific performance problems can be avoided.
In terms of packaging, I can see having 64-bit distros run double-precision Pd for all packages, and 32-bit distros run single precision. That should cover the bulk of situations, the other situations can be covered by custom builds. Having all the 64-bit packages use double-precision Pd is of course going to happen after a while, once we have the bugs worked out. Here I can see an advantage of the monolithic Pd-extended package: its an easy, self-contained test bed.
definitely, the traditional Pd-extended will have an easier time here.
nevertheless, the advent of ~/pd-externals for the user has made things significantly more complicated in terms of "just works".
fgmasdr IOhannes
I don't anticipate any problems with running 64-bit Pd on a 32-bit system, in principle, just as long as the correct data types are set for pointers (same size as t_int) and signals (size of t_sample) differently.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-10-03 18:00, Charles Henry wrote:
Would you prefer to set the types at configure time through a file--or for example by adding a -DDOUBLE compiler flag? The affected locations of code defining the types could just use #ifdef DOUBLE
no, not at all.
In either case, the configure option seems necessary. It still seems an open question how best to offer the double-precision types to externals developers.
yes, of course this is the point. if the external includes (the "correct") m_pd.h, it should get the correct precision for free. right now, the default is to use 32bit. if you set the PD_FLOAT_PRECISION macro to 64 (e.g. by adding "-DPD_FLOAT_PRECISION=64" to the preprocessor-flags or by modifying m_pd.h), you will get a double precision build.
if you set CPPFLAGS, no external will be able to track the current precision. if you modify m_pd.h, then you are modifying upstream sources, which makes it complicated for distribution.
In some cases, the setup() function allocates memory, which needs to be aware of the data type size.
well, just use t_float or t-sample (as appropriate) with the correctly defined PD_FLOAT_PRECISION.
i'm really only talking about how to make sure that PD_FLOAT_PRECISION is defined to the right value.
Adding additional methods seems unnecessary--unless specific performance problems can be avoided.
it's only about guaranteeing consistency between the host (pd) and the client (the external). i don't see so many alternatives, but probably you know some clever trick.
I don't anticipate any problems with running 64-bit Pd on a 32-bit
i'd suggest to use "double precision Pd" (i know it's longer to type) if you refer to 64bit data types, and "64bit Pd" if you refer to address size.
system, in principle, just as long as the correct data types are set for pointers (same size as t_int) and signals (size of t_sample) differently.
that's the problem i'm trying to solve.
fgmasdr IOhannes
On Oct 3, 2011, at 12:00 PM, Charles Henry wrote:
On Mon, Oct 3, 2011 at 10:19 AM, IOhannes m zmoelnig zmoelnig@iem.at wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-10-03 16:31, Hans-Christoph Steiner wrote:
These all sound like good ideas to try. My only concern is that we might let the deployment issues distract from the issues at hand about getting it actually working first.
i'm definitely with you here. what is still missing in terms of "getting it actually working first"?
afaict, katja's patches do make pd itself double-precision ready (the patches might have to be reviewed as far as coding-style is concerned though)
otoh, i wouldn't start "porting" externals before we have a deployment strategy.
one important thing missing right now, is how to compile Pd in a given precision without having to edit m_pd.h technically i think that the define stuff and the like should go into a separate file "types.h" (probably "m_types.h") which is generated from m_types.h.in during configure time, and which is included by m_pd.h (which should remain non-generated) the question is, what miller would think of such a thing.
Would you prefer to set the types at configure time through a file--or for example by adding a -DDOUBLE compiler flag? The affected locations of code defining the types could just use #ifdef DOUBLE
In either case, the configure option seems necessary. It still seems an open question how best to offer the double-precision types to externals developers.
In some cases, the setup() function allocates memory, which needs to be aware of the data type size. Otherwise, memory for signals is allocated through Pd's DSP graph generation routines, so that only changes to externals is to compile perform routines with the given data type.
Adding additional methods seems unnecessary--unless specific performance problems can be avoided.
Wouldn't setting t_sample, t_float, and t_int to 64-bit or 32-bit in m_pd.h combined with sizeof() be enough for this?
.hc
In terms of packaging, I can see having 64-bit distros run double-precision Pd for all packages, and 32-bit distros run single precision. That should cover the bulk of situations, the other situations can be covered by custom builds. Having all the 64-bit packages use double-precision Pd is of course going to happen after a while, once we have the bugs worked out. Here I can see an advantage of the monolithic Pd-extended package: its an easy, self-contained test bed.
definitely, the traditional Pd-extended will have an easier time here.
nevertheless, the advent of ~/pd-externals for the user has made things significantly more complicated in terms of "just works".
fgmasdr IOhannes
I don't anticipate any problems with running 64-bit Pd on a 32-bit system, in principle, just as long as the correct data types are set for pointers (same size as t_int) and signals (size of t_sample) differently.
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
----------------------------------------------------------------------------
"A cellphone to me is just an opportunity to be irritated wherever you are." - Linus Torvalds
On Oct 3, 2011, at 8:28 AM, katja wrote:
On Sun, Oct 2, 2011 at 11:36 PM, Hans-Christoph Steiner hans@at.or.at wrote:
I think it makes sense to work off of pure-data.git rather than pd-extended.git since this is a patch targetted at getting into Miller's repo.
Right. Even then, we could add some external libs to work on, starting with zexy and cyclone for example. The question is how to load appropriate executables into local single and double precision test builds of vanilla Pd. A single preference file is shared by all vanilla Pd installations, therefore setting searchpaths via preferences is no option. Each Pd should only load from it's own 'extra' directory. With the extra's included in vanilla Pd, this works fine as far as I have seen. I tested bonk~ in single and double precision Pd simultaneously without symbol collision.
I think we can pretty rapidly get a double-precision Pd-extended nightly build working, its just that a lot of external objects will be crashy since they use float rather than t_float. I just checked a couple, and they look good in terms of using t_float appropriately.
As for arch issues, I think Intel and ARM are the big ones to test these days. But PPC is fine too.
Expressed in number of 'users', ARM is probably the most popular target hardware for Pd at the moment. It should be easy to patch libpd with the same .patch files, or not? Some modified files are inexistent in libpd (s_audio_pa.c, vexpr.c etc). It's important to at least benchmark-test rewritten code on this hardware indeed, if we want to make a unified doube-precision-ready Pd happen.
In terms of development, let's stick with one codebase. I'll make our jobs easier. Its also easy to compile pure-data.git for ARM, and indeed the 'puredata' package is included in Debian/ARM. Do you have access to an ARM machine? If not, I could probably get one online with ssh access, if that's useful.
.hc
----------------------------------------------------------------------------
Terrorism is not an enemy. It cannot be defeated. It's a tactic. It's about as sensible to say we declare war on night attacks and expect we're going to win that war. We're not going to win the war on terrorism. - retired U.S. Army general, William Odom
On Mon, Oct 3, 2011 at 4:35 PM, Hans-Christoph Steiner hans@at.or.at wrote:
Do you have access to an ARM machine? If not, I could probably get one online with ssh access, if that's useful.
I've mailed Joe White with the question if he can patch the code for libpd and check performance on ARM. He has done some extremely popular RjDj apps and needed to optimize for them as well. Think it would be good anyway to keep in touch with libpd users and app programmers about this topic, even though we're in an early stage with it.
Katja
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-10-03 18:04, katja wrote:
On Mon, Oct 3, 2011 at 4:35 PM, Hans-Christoph Steiner hans@at.or.at wrote:
Do you have access to an ARM machine? If not, I could probably get one online with ssh access, if that's useful.
I've mailed Joe White with the question if he can patch the code for libpd and check performance on ARM.
apropos performance: on my i5 650 @ 3.2GHz, running debian and trying to osc-delay-perfotest.pd (with only 400 osc-delay abstractions, as 500 would max out the CPU in "new double" mode) i get: original : 28% debian : 31% new single: 64% new double: 86%
the "new" versions are Pd-0.43.1-test4 with only the PD_FLOAT_PRECISION set to 32 resp. 64.
the "original" version is a 0.43.1-test2, where i cannot recall any special optimization flags.
the "debian" version is a 0.43.0 with most optimization turned OFF (as is debian policy)
fgmasdr IOhannes
On Oct 3, 2011, at 12:21 PM, IOhannes m zmoelnig wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-10-03 18:04, katja wrote:
On Mon, Oct 3, 2011 at 4:35 PM, Hans-Christoph Steiner hans@at.or.at wrote:
Do you have access to an ARM machine? If not, I could probably get one online with ssh access, if that's useful.
I've mailed Joe White with the question if he can patch the code for libpd and check performance on ARM.
apropos performance: on my i5 650 @ 3.2GHz, running debian and trying to osc-delay-perfotest.pd (with only 400 osc-delay abstractions, as 500 would max out the CPU in "new double" mode) i get: original : 28% debian : 31% new single: 64% new double: 86%
the "new" versions are Pd-0.43.1-test4 with only the PD_FLOAT_PRECISION set to 32 resp. 64.
the "original" version is a 0.43.1-test2, where i cannot recall any special optimization flags.
the "debian" version is a 0.43.0 with most optimization turned OFF (as is debian policy)
Hey IOhannes,
Could you write up a quick wiki page on how to do this performance testing? I never remember a good setup.
.hc
----------------------------------------------------------------------------
I have the audacity to believe that peoples everywhere can have three meals a day for their bodies, education and culture for their minds, and dignity, equality and freedom for their spirits. - Martin Luther King, Jr.
On Mon, Oct 3, 2011 at 6:21 PM, IOhannes m zmoelnig zmoelnig@iem.at wrote:
apropos performance: on my i5 650 @ 3.2GHz, running debian and trying to osc-delay-perfotest.pd (with only 400 osc-delay abstractions, as 500 would max out the CPU in "new double" mode) i get: original : 28% debian : 31% new single: 64% new double: 86%
Did you build new single / new double without any optimization? Makefile.am/in for Pd-0.43.1-test4 do not specify optimization. I compared using -O2 for all builds, like it is set for Pd-0.43-0.
Katja
On Mon, Oct 3, 2011 at 7:08 PM, katja katjavetter@gmail.com wrote:
On Mon, Oct 3, 2011 at 6:21 PM, IOhannes m zmoelnig zmoelnig@iem.at wrote:
apropos performance: on my i5 650 @ 3.2GHz, running debian and trying to osc-delay-perfotest.pd (with only 400 osc-delay abstractions, as 500 would max out the CPU in "new double" mode) i get: original : 28% debian : 31% new single: 64% new double: 86%
Did you build new single / new double without any optimization? Makefile.am/in for Pd-0.43.1-test4 do not specify optimization. I compared using -O2 for all builds, like it is set for Pd-0.43-0.
Katja
Update:
The rewritten code is more sensitive to optimization than the original. On coreduo 1.83 GHz with Debian I could only run 200 osc-delay abstractions in osc-delay-perfotest.pd under worst conditions. Compare these values from command top:
original: 69% with -O0, 47% with -O2 (no SSE) new-single: 83% with -O0, 48% with -O2 (no SSE) new-double: 97% with -O0, 59% with -O2 (no SSE)
On MacBook core2duo 2GHz where I wrote and optimized most, 500 osc-delay abstractions can easily run in osc-delay-perfotest.pd, with these values from top:
original: 60% with -O3 and SSE new-single: 50% with -O3 and SSE new-double: 54% with -O3 and SSE
I knew on beforehand that the code would get tuned (performance-wise) to hardware, instruction set, OS, compiler, compiler options etc. used for development, but it never crossed my mind to check performance with optimization level -O0.
Katja
On Tue, 2011-10-04 at 00:56 +0200, katja wrote:
On Mon, Oct 3, 2011 at 7:08 PM, katja katjavetter@gmail.com wrote:
On Mon, Oct 3, 2011 at 6:21 PM, IOhannes m zmoelnig zmoelnig@iem.at wrote:
apropos performance: on my i5 650 @ 3.2GHz, running debian and trying to osc-delay-perfotest.pd (with only 400 osc-delay abstractions, as 500 would max out the CPU in "new double" mode) i get: original : 28% debian : 31% new single: 64% new double: 86%
Did you build new single / new double without any optimization? Makefile.am/in for Pd-0.43.1-test4 do not specify optimization. I compared using -O2 for all builds, like it is set for Pd-0.43-0.
Katja
Update:
The rewritten code is more sensitive to optimization than the original. On coreduo 1.83 GHz with Debian I could only run 200 osc-delay abstractions in osc-delay-perfotest.pd under worst conditions. Compare these values from command top:
original: 69% with -O0, 47% with -O2 (no SSE) new-single: 83% with -O0, 48% with -O2 (no SSE) new-double: 97% with -O0, 59% with -O2 (no SSE)
On MacBook core2duo 2GHz where I wrote and optimized most, 500 osc-delay abstractions can easily run in osc-delay-perfotest.pd, with these values from top:
original: 60% with -O3 and SSE new-single: 50% with -O3 and SSE new-double: 54% with -O3 and SSE
I knew on beforehand that the code would get tuned (performance-wise) to hardware, instruction set, OS, compiler, compiler options etc. used for development, but it never crossed my mind to check performance with optimization level -O0.
I think this is the right approach to coding this stuff. Both the compilers and the CPUs can do a lot more since that code was originally written.
Debian doesn't compile things at -O0, but it does not allow CPU-specific optimizations (i.e. SSE). I'm sure -O2 is ok for Debian. aI hope that Debian allows SSE on the amd64 arch, since all x86_64 CPUs support SSE. SSE has been in Intel i386 chips since 1999 and I think basically all i386 chips since 2004.
And we have our first Pd-double build! http://autobuild.puredata.info/auto-build/2011-10-03/Pd-0.43.1-double-201110...
rsync -av --delete rsync://128.238.56.50/distros/pd-double/ pd-double/
.hc
On Tue, Oct 4, 2011 at 1:33 AM, Hans-Christoph Steiner hans@at.or.at wrote:
And we have our first Pd-double build! http://autobuild.puredata.info/auto-build/2011-10-03/Pd-0.43.1-double-201110... macosx106-x86_64.dmg
Cool Hans! Can't wait to check it out (though I'll have to wait till I get home tonight).
Yesterday I forgot to mention why it should definitely not be built with -O0 (unless for debug purposes): PD_BIGORSMALL is defined an inline function (like it was already suggested by IOhannes a while ago), but at -O0 nothing will be inlined. A benchmark howto would be useful indeed.
By the way for my coreduo 1.83 GHZ I could compile for Debian with SSE by setting -march=prescott, this was the last 32bit adress space SSE enabled CPU.
Katja
On Tue, Oct 4, 2011 at 9:06 AM, katja katjavetter@gmail.com wrote:
By the way for my coreduo 1.83 GHZ I could compile for Debian with SSE by setting -march=prescott, this was the last 32bit adress space SSE enabled CPU.
Sorry, correction again: of course coreduo is also 32bit address space but it is not in the list of possible arch definitions for gcc.
Katja
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-10-04 09:06, katja wrote:
Yesterday I forgot to mention why it should definitely not be built with -O0 (unless for debug purposes): PD_BIGORSMALL is defined an
ah yes, this was indeed my fault. since i don't feel comfortable with editing m_pd.h to get a different build, i used CFLAGS="-DPD_FLOAT_PRECISION=64", which undid any optimization flags (which by default are "-O6", which i find a bit overdone; and "-g" is not set at all...)
the proper way is to use CPPFLAGS="-DPD_FLOAT_PRECISION=64", which results in:
osc-delay-perftest with 400 instances: debian : 31% original : 29% single : 22% single(O0) : 64% single(O2) : 25% single(O2+loop) : 22% single(pentium3) : 24% single(pentium4) : 22% single(prescott) : 22% single(core2) : 22% single(core2+sse): 22% double : 25% double(O0) : 86% double(O2) : 27% double(O2+loop) : 26% double(pentium3) : 25% double(pentium4) : 24% double(prescott) : 24% double(core2) : 24% double(core2+sse): 25%
osc-delay-perftest with 1200 instances: debian : 94% original : 81% single : 65% single(O2) : 72% single(O0) : ++% single(O2+loop) : 66% single(pentium3) : 70% single(pentium4) : 66% single(prescott) : 65% single(core2) : 59% single(core2+sse): 64% double : 77% double(O0) : ++% double(O2) : 82% double(O2+loop) : 77% double(pentium3) : 79% double(pentium4) : 75% double(prescott) : 75% double(core2) : 71% double(core2+sse): 75%
which is more inline with katja's measurements.
this is (again) on an i5 650 @ 3.2GHz running in 32bit mode optimization flags (as far as they can be reconstructed :-)) debian: "-g -O2" (this is what is dictated by debian policy) original: "-O6 -funroll-loops -fomit-frame-pointer" (seems to be the default) single/double: ->original (O0): -O0 (O2): -g -O2 (O2+loop): -g -O2 -funroll-loops -fomit-frame-pointer (prescott): ->original + "-march=prescott" (core2): ->original + "-march=core2" (core2+sse): ->original + "-march=core2 -mfpmath=sse -msse2"
so it seems like the biggest performance boost is given (on the tested platform), by compiling with "-g -O2 -funroll-loops - -fomit-frame-pointer" (which is cool because i think this can even make it into debian, the way it is)
inline function (like it was already suggested by IOhannes a while ago), but at -O0 nothing will be inlined. A benchmark howto would be useful indeed.
well, i usually just cram lots of the same object into a subpatch (until i get approximately 80% in the slowest environment, in order to not max out the CUP and get unknown side-effects), and measure it with the built-in load-meter (for loads <100% it behaves quite the same as top) nothing very dramatic.
fgmasdr IOhannes
On Oct 4, 2011, at 5:38 AM, IOhannes m zmoelnig wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-10-04 09:06, katja wrote:
Yesterday I forgot to mention why it should definitely not be built with -O0 (unless for debug purposes): PD_BIGORSMALL is defined an
ah yes, this was indeed my fault. since i don't feel comfortable with editing m_pd.h to get a different build, i used CFLAGS="-DPD_FLOAT_PRECISION=64", which undid any optimization flags (which by default are "-O6", which i find a bit overdone; and "-g" is not set at all...)
the proper way is to use CPPFLAGS="-DPD_FLOAT_PRECISION=64", which results in:
osc-delay-perftest with 400 instances: debian : 31% original : 29% single : 22% single(O0) : 64% single(O2) : 25% single(O2+loop) : 22% single(pentium3) : 24% single(pentium4) : 22% single(prescott) : 22% single(core2) : 22% single(core2+sse): 22% double : 25% double(O0) : 86% double(O2) : 27% double(O2+loop) : 26% double(pentium3) : 25% double(pentium4) : 24% double(prescott) : 24% double(core2) : 24% double(core2+sse): 25%
osc-delay-perftest with 1200 instances: debian : 94% original : 81% single : 65% single(O2) : 72% single(O0) : ++% single(O2+loop) : 66% single(pentium3) : 70% single(pentium4) : 66% single(prescott) : 65% single(core2) : 59% single(core2+sse): 64% double : 77% double(O0) : ++% double(O2) : 82% double(O2+loop) : 77% double(pentium3) : 79% double(pentium4) : 75% double(prescott) : 75% double(core2) : 71% double(core2+sse): 75%
which is more inline with katja's measurements.
this is (again) on an i5 650 @ 3.2GHz running in 32bit mode optimization flags (as far as they can be reconstructed :-)) debian: "-g -O2" (this is what is dictated by debian policy) original: "-O6 -funroll-loops -fomit-frame-pointer" (seems to be the default) single/double: ->original (O0): -O0 (O2): -g -O2 (O2+loop): -g -O2 -funroll-loops -fomit-frame-pointer (prescott): ->original + "-march=prescott" (core2): ->original + "-march=core2" (core2+sse): ->original + "-march=core2 -mfpmath=sse -msse2"
so it seems like the biggest performance boost is given (on the tested platform), by compiling with "-g -O2 -funroll-loops
- -fomit-frame-pointer" (which is cool because i think this can even
make it into debian, the way it is)
inline function (like it was already suggested by IOhannes a while ago), but at -O0 nothing will be inlined. A benchmark howto would be useful indeed.
well, i usually just cram lots of the same object into a subpatch (until i get approximately 80% in the slowest environment, in order to not max out the CUP and get unknown side-effects), and measure it with the built-in load-meter (for loads <100% it behaves quite the same as top) nothing very dramatic.
Nice tests, thanks for that. I would be interested to see the effects of auto-vectorization on these numbers. Have you tried that? If the test patch doesn't include objects that have loops vectorized, it won't make a difference.
.hc
----------------------------------------------------------------------------
If you are not part of the solution, you are part of the problem.
On Tue, Oct 4, 2011 at 11:38 AM, IOhannes m zmoelnig zmoelnig@iem.at wrote:
the proper way is to use CPPFLAGS="-DPD_FLOAT_PRECISION=64",
But now you undo the CPPFLAGS as defined in the makefile. I didn't know how to add to the CFLAGS from the command line, but found a solution here:
http://theory.uwinnipeg.ca/localfiles/infofiles/make/make_66.html#SEC65
It requires a small adaptation to the makefile. Instead of:
CFLAGS = ....
comes:
override CFLAGS += .....
Now you can add to CFLAGS from the command line, like so:
make CFLAGS="-DPD_FLOAT_PRECISION=64"
Note that the CFLAGS in the makefile now have precedence and you can only add to it from the command line, not override it.
Katja
On Oct 5, 2011, at 6:40 AM, katja wrote:
On Tue, Oct 4, 2011 at 11:38 AM, IOhannes m zmoelnig zmoelnig@iem.at wrote:
the proper way is to use CPPFLAGS="-DPD_FLOAT_PRECISION=64",
But now you undo the CPPFLAGS as defined in the makefile. I didn't know how to add to the CFLAGS from the command line, but found a solution here:
http://theory.uwinnipeg.ca/localfiles/infofiles/make/ make_66.html#SEC65
It requires a small adaptation to the makefile. Instead of:
CFLAGS = ....
comes:
override CFLAGS += .....
Now you can add to CFLAGS from the command line, like so:
make CFLAGS="-DPD_FLOAT_PRECISION=64"
Note that the CFLAGS in the makefile now have precedence and you can only add to it from the command line, not override it.
For the sake of this dev branch, I think we can have it automatically detect 32-bit vs. 64-bit platforms, and set accordingly. Does that work for people?
.hc
----------------------------------------------------------------------------
If you are not part of the solution, you are part of the problem.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 10/05/2011 12:40 PM, katja wrote:
On Tue, Oct 4, 2011 at 11:38 AM, IOhannes m zmoelnig zmoelnig@iem.at wrote:
the proper way is to use CPPFLAGS="-DPD_FLOAT_PRECISION=64",
But now you undo the CPPFLAGS as defined in the makefile. I didn't know how to add to the CFLAGS from the command line, but found a solution here:
http://theory.uwinnipeg.ca/localfiles/infofiles/make/make_66.html#SEC65
It requires a small adaptation to the makefile. Instead of:
CFLAGS = ....
comes:
override CFLAGS += .....
Now you can add to CFLAGS from the command line, like so:
make CFLAGS="-DPD_FLOAT_PRECISION=64"
Note that the CFLAGS in the makefile now have precedence and you can only add to it from the command line, not override it.
i don't get the point here, since CPPFLAGS is not set in the Pd Makefiles, so setting them to outside should have no weird sideeffects, whereas CFLAGS does.
ah, on closer inspection it seems like you are building Pd with the old autoconf system (pd/src/configure.ac generates pd/src/makefile), whereas i am using (and talking about) the newer autotools based build system (pd/configure.ac generates pd/src/Makefile)
afaik, this is the build-system used in the nightly builds (apart from w32)
fgmasdr IOhannes
On Wed, 2011-10-05 at 21:37 +0200, IOhannes m zmölnig wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 10/05/2011 12:40 PM, katja wrote:
On Tue, Oct 4, 2011 at 11:38 AM, IOhannes m zmoelnig zmoelnig@iem.at wrote:
the proper way is to use CPPFLAGS="-DPD_FLOAT_PRECISION=64",
But now you undo the CPPFLAGS as defined in the makefile. I didn't know how to add to the CFLAGS from the command line, but found a solution here:
http://theory.uwinnipeg.ca/localfiles/infofiles/make/make_66.html#SEC65
It requires a small adaptation to the makefile. Instead of:
CFLAGS = ....
comes:
override CFLAGS += .....
Now you can add to CFLAGS from the command line, like so:
make CFLAGS="-DPD_FLOAT_PRECISION=64"
Note that the CFLAGS in the makefile now have precedence and you can only add to it from the command line, not override it.
i don't get the point here, since CPPFLAGS is not set in the Pd Makefiles, so setting them to outside should have no weird sideeffects, whereas CFLAGS does.
ah, on closer inspection it seems like you are building Pd with the old autoconf system (pd/src/configure.ac generates pd/src/makefile), whereas i am using (and talking about) the newer autotools based build system (pd/configure.ac generates pd/src/Makefile)
afaik, this is the build-system used in the nightly builds (apart from w32)
Yeah, the nightly builds look for pd/autogen.sh and if its found, use that. Windows/MinGW can build using that build system, but its not entirely working yet, so the nightly builds still use pd/src/makefile.mingw.
I removed the old build system from pd-double.git and pushed the change. Hopefully that'll reduce confusion.
.hc
2011/10/5 Hans-Christoph Steiner hans@at.or.at:
I removed the old build system from pd-double.git and pushed the change. Hopefully that'll reduce confusion.
Yeah I was using the old build system all the time because it was so easy to produce local builds by doing make without install. Never mind.
Katja.
On Thu, 2011-10-06 at 00:33 +0200, katja wrote:
2011/10/5 Hans-Christoph Steiner hans@at.or.at:
I removed the old build system from pd-double.git and pushed the change. Hopefully that'll reduce confusion.
Yeah I was using the old build system all the time because it was so easy to produce local builds by doing make without install. Never mind.
I make and run local builds all of the time for testing, it is indeed easy. The new build system should allow for that too. But yeah, you do need to set it up by doing:
./autogen.sh && ./configure
Then it should work for a while. I just pushed a change to pd/configure.ac in pd-double to enable detection of 64-bit platforms, and enable double-precision if on 64-bit. I also added:
./configure --enable-double-precision
to force the use of double precision on any platform, or
./configure --disable-double-precision
to force the use of single precision on any platform.
.hc
On Oct 3, 2011, at 12:04 PM, katja wrote:
On Mon, Oct 3, 2011 at 4:35 PM, Hans-Christoph Steiner hans@at.or.at wrote:
Do you have access to an ARM machine? If not, I could probably get one online with ssh access, if that's useful.
I've mailed Joe White with the question if he can patch the code for libpd and check performance on ARM. He has done some extremely popular RjDj apps and needed to optimize for them as well. Think it would be good anyway to keep in touch with libpd users and app programmers about this topic, even though we're in an early stage with it.
Yes definitely, we should let everyone who wants to be get involved. I am just saying with need a development platform to start with. Once that's nailed down, we can deal with more issues, like porting to libpd, dealing with externals that could be either 32-bit or 64-bit, etc.
I setup a nightly build on the macosx106-x86_64 and called it pd- double. Andras and r33p, if you are listening, could you run this build on your 64-bit boxes also? All you need to do is:
~pd/auto-build cp -a pd-extended pd-double
.hc
----------------------------------------------------------------------------
"Free software means you control what your computer does. Non-free software means someone else controls that, and to some extent controls you." - Richard M. Stallman
On Mon, Oct 3, 2011 at 18:26, Hans-Christoph Steiner hans@at.or.at wrote:
On Oct 3, 2011, at 12:04 PM, katja wrote:
On Mon, Oct 3, 2011 at 4:35 PM, Hans-Christoph Steiner hans@at.or.at
wrote:
Do you have access to an ARM
machine? If not, I could probably get one online with ssh access, if that's useful.
I've mailed Joe White with the question if he can patch the code for libpd and check performance on ARM. He has done some extremely popular RjDj apps and needed to optimize for them as well. Think it would be good anyway to keep in touch with libpd users and app programmers about this topic, even though we're in an early stage with it.
Yes definitely, we should let everyone who wants to be get involved. I am just saying with need a development platform to start with. Once that's nailed down, we can deal with more issues, like porting to libpd, dealing with externals that could be either 32-bit or 64-bit, etc.
I setup a nightly build on the macosx106-x86_64 and called it pd-double. Andras and r33p, if you are listening, could you run this build on your 64-bit boxes also? All you need to do is:
~pd/auto-build cp -a pd-extended pd-double
Listening now. I did: $ cd ~pd/auto-build $ sudo cp -a pd-extended pd-double What's next? Shall I try patching or rather pull IOhannes's sources?
Andras
On Oct 4, 2011, at 10:19 AM, András Murányi wrote:
On Mon, Oct 3, 2011 at 18:26, Hans-Christoph Steiner hans@at.or.at wrote:
On Oct 3, 2011, at 12:04 PM, katja wrote:
On Mon, Oct 3, 2011 at 4:35 PM, Hans-Christoph Steiner hans@at.or.at wrote:
Do you have access to an ARM machine? If not, I could probably get one online with ssh access, if that's useful.
I've mailed Joe White with the question if he can patch the code for libpd and check performance on ARM. He has done some extremely popular RjDj apps and needed to optimize for them as well. Think it would be good anyway to keep in touch with libpd users and app programmers about this topic, even though we're in an early stage with it.
Yes definitely, we should let everyone who wants to be get involved. I am just saying with need a development platform to start with. Once that's nailed down, we can deal with more issues, like porting to libpd, dealing with externals that could be either 32-bit or 64-bit, etc.
I setup a nightly build on the macosx106-x86_64 and called it pd- double. Andras and r33p, if you are listening, could you run this build on your 64-bit boxes also? All you need to do is:
~pd/auto-build cp -a pd-extended pd-double
Listening now. I did: $ cd ~pd/auto-build $ sudo cp -a pd-extended pd-double What's next? Shall I try patching or rather pull IOhannes's sources?
If you have the run-automated-builder script in a cron job, that is all you have to do.
.hc
----------------------------------------------------------------------------
¡El pueblo unido jamás será vencido!
2011/10/4 Hans-Christoph Steiner hans@at.or.at
On Oct 4, 2011, at 10:19 AM, András Murányi wrote:
On Mon, Oct 3, 2011 at 18:26, Hans-Christoph Steiner hans@at.or.atwrote:
On Oct 3, 2011, at 12:04 PM, katja wrote:
On Mon, Oct 3, 2011 at 4:35 PM, Hans-Christoph Steiner hans@at.or.at
wrote:
Do you have access to an ARM
machine? If not, I could probably get one online with ssh access, if that's useful.
I've mailed Joe White with the question if he can patch the code for libpd and check performance on ARM. He has done some extremely popular RjDj apps and needed to optimize for them as well. Think it would be good anyway to keep in touch with libpd users and app programmers about this topic, even though we're in an early stage with it.
Yes definitely, we should let everyone who wants to be get involved. I am just saying with need a development platform to start with. Once that's nailed down, we can deal with more issues, like porting to libpd, dealing with externals that could be either 32-bit or 64-bit, etc.
I setup a nightly build on the macosx106-x86_64 and called it pd-double. Andras and r33p, if you are listening, could you run this build on your 64-bit boxes also? All you need to do is:
~pd/auto-build cp -a pd-extended pd-double
Listening now. I did: $ cd ~pd/auto-build $ sudo cp -a pd-extended pd-double What's next? Shall I try patching or rather pull IOhannes's sources?
If you have the run-automated-builder script in a cron job, that is all you have to do.
.hc
Ah, so tomorrow a single and double precision build will automatically be made? Cool.
Also, as I was busy with my life (buying a flat) these days, and I couldn't follow the list as precisely as I wished, could you advise me what's the current best way to roll my own double precision pd? Because I would like to benchmark a fully optimised one.
Thanks,
Andras
On Oct 4, 2011, at 10:54 AM, András Murányi wrote:
2011/10/4 Hans-Christoph Steiner hans@at.or.at
On Oct 4, 2011, at 10:19 AM, András Murányi wrote:
On Mon, Oct 3, 2011 at 18:26, Hans-Christoph Steiner hans@at.or.at wrote:
On Oct 3, 2011, at 12:04 PM, katja wrote:
On Mon, Oct 3, 2011 at 4:35 PM, Hans-Christoph Steiner hans@at.or.at wrote:
Do you have access to an ARM machine? If not, I could probably get one online with ssh access, if that's useful.
I've mailed Joe White with the question if he can patch the code for libpd and check performance on ARM. He has done some extremely popular RjDj apps and needed to optimize for them as well. Think it would be good anyway to keep in touch with libpd users and app programmers about this topic, even though we're in an early stage with it.
Yes definitely, we should let everyone who wants to be get involved. I am just saying with need a development platform to start with. Once that's nailed down, we can deal with more issues, like porting to libpd, dealing with externals that could be either 32-bit or 64-bit, etc.
I setup a nightly build on the macosx106-x86_64 and called it pd- double. Andras and r33p, if you are listening, could you run this build on your 64-bit boxes also? All you need to do is:
~pd/auto-build cp -a pd-extended pd-double
Listening now. I did: $ cd ~pd/auto-build $ sudo cp -a pd-extended pd-double What's next? Shall I try patching or rather pull IOhannes's sources?
If you have the run-automated-builder script in a cron job, that is all you have to do.
.hc
Ah, so tomorrow a single and double precision build will automatically be made? Cool.
Also, as I was busy with my life (buying a flat) these days, and I couldn't follow the list as precisely as I wished, could you advise me what's the current best way to roll my own double precision pd? Because I would like to benchmark a fully optimised one.
That would great to have those numbers. I just committed some changes to set lots of optimization flags, since all of the build servers are using gcc 4.x now. So looking at this commit will show you the place to set the optimization flags:
http://pure-data.svn.sourceforge.net/viewvc/pure-data?view=revision&revi...
'make clean' in the various packages/* folders should work, but I haven't throughly tested it, and use the rsync in the script to be sure.
.hc
----------------------------------------------------------------------------
Access to computers should be unlimited and total. - the hacker ethic
Hello,
Happy to see so many test results from IOhannes.
The 'perfotest' patches were initially created for function profiling, to check if there are particularly time consuming instructions. To mention a funny example: I was happy to see that fabs() was translated to a single instruction ANDPS / ANDPD for xmm registers. But for the FPU it is a call. The same for isnan(). That's why PD_BIGORSMALL must still do a bitpattern check on aliased floats.
For benchmarking original and double-ready Pd as a whole, I used two (fairly cool and elaborate) patches which were written in pure vanilla:
- Chaosmonster1 from www.martin-brinkmann.de (10 instances or so) - Cave of Creation by Hamster, http://puredata.hurleur.com/sujet-5080-2.html
Both works feature enough of the rewritten code to make them representative overall benchmarks. If double-ready Pd performs as well as original Pd with usual compiler settings, on all possible platforms, I would be satisfied. After all, the purpose of the whole thing was to get some more decimal places in our numbers, not to make Pd run faster.
Katja
Forgot to mention this: at start up there's a logpost (level 3) 'PD_FLOATPRECISION=32 bits' for single and 'PD_FLOATPRECISION=64 bits' for double build.
Ah, so tomorrow a single and double precision build will automatically be made? Cool.
It's confusing. At the moment there is vanilla Pd patched to work in double precision. But for Pd-extended it is: a single precision Pd-extended with double-ready core code. Not a double precision Pd-extended, not even double-ready Pd-extended. Let's better call it something like Pd-0.43.1-single-20111004-macosx106-x86_64.dmg etc., Hans?
Katja
On Oct 4, 2011, at 7:06 PM, katja wrote:
Forgot to mention this: at start up there's a logpost (level 3) 'PD_FLOATPRECISION=32 bits' for single and 'PD_FLOATPRECISION=64 bits' for double build.
Ah, so tomorrow a single and double precision build will automatically be made? Cool.
It's confusing. At the moment there is vanilla Pd patched to work in double precision. But for Pd-extended it is: a single precision Pd-extended with double-ready core code. Not a double precision Pd-extended, not even double-ready Pd-extended. Let's better call it something like Pd-0.43.1-single-20111004-macosx106-x86_64.dmg etc., Hans?
Its a dev branch to test the double stuff, so its going to be messy, unless someone wants to clean up the scripts. Pd-extended is still using 32-bit floats for t_float and t_symbol. The pd-double build will have some vestiges of the 'extended' name in it, because the build scripts are crufty and kludgey and should be replaced. But they work.
So you are saying that the stuff in pd-double is not building using 64- bit floats? Let's get a github repo going so we can work on this stuff. Unless you want to, I'll happily set one up at:
https://github.com/pd-projects/pd-double
And add people who are interested in working on it. Or do you want to maintain your own git repo that we submit patches to?
.hc
----------------------------------------------------------------------------
You can't steal a gift. Bird gave the world his music, and if you can hear it, you can have it. - Dizzy Gillespie
-
András Murányi -
Charles Henry -
Hans-Christoph Steiner -
IOhannes m zmoelnig -
IOhannes m zmölnig
-
katja