After almost constant fighting with nastiness like denormal numbers and unexplained CPU spikes with PD under Linux on a P4 processor, I have decided to look into recompiling it using ICC. I reached this conclusion on that grounds that PD never ran so terribly before when I used windoze, and it runs quite nice with an Athlon processor under Linux, so I figure it must be a compiler issue.
Can anyone give me a set of recommended flags, or any other helpful tips, for doing this? I'm still stuck trying to figure out how to get ICC working [+ waiting for stupid fscking 'registration key' to arrive via email... ugh!!!!]
thx, d.
After almost constant fighting with nastiness like denormal numbers and unexplained CPU spikes with PD under Linux on a P4 processor, I have decided to look into recompiling it using ICC. I reached this conclusion on that grounds that PD never ran so terribly before when I used windoze, and it runs quite nice with an Athlon processor under Linux, so I figure it must be a compiler issue.
i'm not completely shure how icc for ia32 can handle denormals ... their 64bit compiler has a flag to flush denormals to zero, but not the ia32 compiler...
well, there is pd's PD_BADFLOAT macro that _should_ to that, too, but this has only effect if the nubers are already denormal, so maybe it's a good idea to reset the value to flush every number a 24 / 32 bit dac would consider to be zero to zero...
Can anyone give me a set of recommended flags, or any other helpful tips, for doing this? I'm still stuck trying to figure out how to get ICC working [+ waiting for stupid fscking 'registration key' to arrive
if you are using the current devel branch from cvs, just configure with --enable-icc --enable-optimize=pentium4 ... i didn't wanted to edit my makefiles every time...
cheers...
Tim mailto:TimBlechmann@gmx.de ICQ: 96771783 -- The only people for me are the mad ones, the ones who are mad to live, mad to talk, mad to be saved, desirous of everything at the same time, the ones who never yawn or say a commonplace thing, but burn, burn, burn, like fabulous yellow roman candles exploding like spiders across the stars and in the middle you see the blue centerlight pop and everybody goes "Awww!" Jack Kerouac
Hi Tim,
Tim Blechmann wrote:
i'm not completely shure how icc for ia32 can handle denormals ... their 64bit compiler has a flag to flush denormals to zero, but not the ia32 compiler...
What are the maintainers of the windoze packages of PD using to compile? Whatever it is, it seems to clear up any denormal or CPU spike problems.
well, there is pd's PD_BADFLOAT macro that _should_ to that, too, but this has only effect if the nubers are already denormal, so maybe it's a good idea to reset the value to flush every number a 24 / 32 bit dac would consider to be zero to zero...
Can you tell me a bit more about this? How and where should I start with that?
I have found that certain objects have *stong* denormal tendencies, including hip~, lp~, np~, throw~, catch~ and probably more that I haven't weeded out yet.
The bottom line is that patches which are completely fine to run on windoze and on most AMD processors with Linux completely screw up my P4 laptop. We are talking grinding halts of DSP, sudden [+ lasting!] leaps of CPU usage over 100% and even completely freezing the machine if I happen to be in realtime mode. Not pretty.
This has made me very careful about my patches these days, and while it has improved my patching technique immensely ;-), it is still a real problem when I am the only one getting errors with my own patches, and everyone else who tests them in a different environment has none at all.
I've written this list about it several times now, but I am not a C programmer, so there are only limited things which I can do [besides buying a new laptop or switching back to winschnoz]. Trying to recompile with a different compiler is one of them.
if you are using the current devel branch from cvs, just configure with --enable-icc --enable-optimize=pentium4 ... i didn't wanted to edit my makefiles every time...
Using "./configure --enable-icc --enable-optimize=pentium4 --enable-jack --enable-fftw" still uses GCC. Using "CC=icc CXX=icc" tells it to use ICC, but I get an error that the C compiler cannot create executables. I have a hunch that this is because I still haven't gotten this ridiculous key ["It should arrive within one business day", sez Intel...]
I [heart] computers. d.
hi derek...
What are the maintainers of the windoze packages of PD using to compile? Whatever it is, it seems to clear up any denormal or CPU spike problems.
just a guess, but maybe the linux system libraries create more denormals than the windows ones ...
well, there is pd's PD_BADFLOAT macro that _should_ to that, too, but this has only effect if the nubers are already denormal, so maybe it's a good idea to reset the value to flush every number a 24 / 32 bit dac would consider to be zero to zero...
Can you tell me a bit more about this? How and where should I start with that?
the PD_BADFLOAT macro is currently just checking for actual denormals... denormals occur at about 1e-34 ... a 32 bit dac should have a resolution of about 5e-10 ... so i suppose we could implement a macro that flushes everything below 1e-12 to zero... the best would be to find a fancy mask, to detect these numbers ... well, you'll have to mess around with mantissa ... if you could have a look at this it would be great...
I have found that certain objects have *stong* denormal tendencies, including hip~, lp~, np~, throw~, catch~ and probably more that I haven't weeded out yet.
well, digital filters and other delay lines are dangerous about denormals, but i'm a bit confused about the throw~ / catch~ objects ... i haven't had a look at the code, but to me it seems they just copy sample arrays ...
The bottom line is that patches which are completely fine to run on windoze and on most AMD processors with Linux completely screw up my P4 laptop. We are talking grinding halts of DSP, sudden [+ lasting!] leaps of CPU usage over 100% and even completely freezing the machine if I happen to be in realtime mode. Not pretty.
i know the problem, tooo :-(
Using "./configure --enable-icc --enable-optimize=pentium4 --enable-jack --enable-fftw" still uses GCC. Using "CC=icc CXX=icc" tells it to use ICC, but I get an error that the C compiler cannot create executables. I have a hunch that this is because I still haven't gotten this ridiculous key ["It should arrive within one business day", sez Intel...]
you shouldn't have to set the CC or CXX variables ... i'll have a look at it ... about the configure message ... is you icc executable in the /usr/bin path? if not, you'll probably have to create a symlink to it...
cheers...
Tim mailto:TimBlechmann@gmx.de ICQ: 96771783 -- The only people for me are the mad ones, the ones who are mad to live, mad to talk, mad to be saved, desirous of everything at the same time, the ones who never yawn or say a commonplace thing, but burn, burn, burn, like fabulous yellow roman candles exploding like spiders across the stars and in the middle you see the blue centerlight pop and everybody goes "Awww!" Jack Kerouac
Tim,
Tim Blechmann wrote:
you shouldn't have to set the CC or CXX variables ... i'll have a look at it ... about the configure message ... is you icc executable in the /usr/bin path? if not, you'll probably have to create a symlink to it...
My ICC is from Gentoo, which puts it in /opt. I symlinked
/usr/bin/icc -> /opt/intel/compiler80/bin/icc
but it still starts with GCC. Also tried various other combinations of directories etc, but still the same. The only way to get it to use ICC is with CC or CXX flags.
d.
but it still starts with GCC. Also tried various other combinations of directories etc, but still the same. The only way to get it to use ICC is with CC or CXX flags.
could you send me the output of ./configure? maybe you have to run autoconf again ... (just a guess)
i hope i can say more, later ...
Tim mailto:TimBlechmann@gmx.de ICQ: 96771783 -- The only people for me are the mad ones, the ones who are mad to live, mad to talk, mad to be saved, desirous of everything at the same time, the ones who never yawn or say a commonplace thing, but burn, burn, burn, like fabulous yellow roman candles exploding like spiders across the stars and in the middle you see the blue centerlight pop and everybody goes "Awww!" Jack Kerouac
Tim,
I compiled perfectly fine with ICC using CC=icc and CXX=icc flags. I will start testing to see if it fixes some of my problems tonight, and maybe try some other flags for different optimizations. Results to this list soon.
Tim Blechmann wrote:
but it still starts with GCC. Also tried various other combinations of directories etc, but still the same. The only way to get it to use ICC is with CC or CXX flags.
could you send me the output of ./configure? maybe you have to run autoconf again ... (just a guess)
It is a completely normal configure, except that it ignores ICC and uses GCC. See for yourself below. I even tried --disable-gcc, and it ignored that. The two flags you mention: --enable-optimize=pentium4 and --enable-icc do *not* appear when I type ./configure --help, although many others do.
I also recall that I could not set the OPT_CFLAGS in the ./configure stage, I actually had to enter them manually in the "configure" file. According to ./configure --help, this should be possible.
Of course, I am still just a voodoo programmer, throwing flags and variables around without ever being totally sure what I am doing, so it could be I missed something important alongh the way!
root 18:39:41> ./configure --enable-jack --enable-fftw --enable-icc --enable-optimize=pentium4 checking for gcc... gcc checking for C compiler default output file name... a.out checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ANSI C... none needed checking for a BSD-compatible install... /bin/install -c checking whether make sets $(MAKE)... yes checking how to run the C preprocessor... gcc -E checking for an ANSI C-conforming const... yes checking for egrep... grep -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking for pid_t... yes checking for size_t... yes checking whether time.h and sys/time.h may both be included... yes checking for ANSI C header files... (cached) yes checking fcntl.h usability... yes checking fcntl.h presence... yes checking for fcntl.h... yes checking limits.h usability... yes checking limits.h presence... yes checking for limits.h... yes checking malloc.h usability... yes checking malloc.h presence... yes checking for malloc.h... yes checking sys/ioctl.h usability... yes checking sys/ioctl.h presence... yes checking for sys/ioctl.h... yes checking sys/time.h usability... yes checking sys/time.h presence... yes checking for sys/time.h... yes checking for unistd.h... (cached) yes checking bstring.h usability... no checking bstring.h presence... no checking for bstring.h... no checking whether gcc needs -traditional... no checking return type of signal handlers... void checking for vprintf... yes checking for _doprnt... no checking for gettimeofday... yes checking for select... yes checking for socket... yes checking for strerror... yes checking for dlopen in -ldl... yes checking for sin in -lm... yes checking for pthread_create in -lpthread... yes checking for X... libraries /usr/X11R6/lib, headers checking for XCreateWindow in -lX11... yes checking for main in -ltcl8.7... no checking for main in -ltcl8.6... no checking for main in -ltcl8.5... no checking for main in -ltcl8.4... yes checking for main in -ltk8.4... yes checking tcl.h usability... yes checking tcl.h presence... yes checking for tcl.h... yes yes ... alsa is... yes checking for snd_pcm_info in -lasound... yes checking for shm_open in -lrt... yes checking for jack_set_xrun_callback in -ljack... yes checking for jack_set_error_function in -ljack... yes checking for fftwf_plan_dft_r2c_1d in -lfftw3f... yes OPT_CFLAGS --------------- -O2 -pipe -fomit-frame-pointer -march=pentium4 configure: creating ./config.status config.status: creating makefile
[Please note that the OPT_CFLAGS above was hand-coded into 'configure'. I forgot to remove it just now, and I can't exactly remember what was there before, anyway ;-) But curiously, I don't see any sign of --enable-optimize=pentium4 anywhere here, nor in the output of 'make'!]
cheers, d.
It is a completely normal configure, except that it ignores ICC and uses GCC. See for yourself below. I even tried --disable-gcc, and it ignored that. The two flags you mention: --enable-optimize=pentium4 and --enable-icc do *not* appear when I type ./configure --help, although many others do.
well, it seems that my changes to the configure script haven't reached you ... are you on the recent cvs (devel_0_37)???
I also recall that I could not set the OPT_CFLAGS in the ./configure stage, I actually had to enter them manually in the "configure" file. According to ./configure --help, this should be possible.
btw, there is an explanation on the denormal issue at: http://www.musicdsp.org/phpWiki/index.php/What%20is%20a%20denormal%20number%...
cheers...
Tim mailto:TimBlechmann@gmx.de ICQ: 96771783 -- The only people for me are the mad ones, the ones who are mad to live, mad to talk, mad to be saved, desirous of everything at the same time, the ones who never yawn or say a commonplace thing, but burn, burn, burn, like fabulous yellow roman candles exploding like spiders across the stars and in the middle you see the blue centerlight pop and everybody goes "Awww!" Jack Kerouac
derek holzer wrote:
Hi Tim,
Tim Blechmann wrote:
i'm not completely shure how icc for ia32 can handle denormals ... their 64bit compiler has a flag to flush denormals to zero, but not the ia32 compiler...
What are the maintainers of the windoze packages of PD using to compile? Whatever it is, it seems to clear up any denormal or CPU spike problems.
i have gotten denormal cpu spikes from the MSP version of pd. which version have you used that doesnt have them?
Josh Steiner wrote:
i have gotten denormal cpu spikes from the MSP version of pd. which version have you used that doesnt have them?
You mean for win, right?
Don't recall which versions I was using. Haven't booted windoze in over a year. But I am pretty sure it was PD extended, from Sourceforge, running on win2K, although exact version I am unsure of. If it was a year or so ago, that would mean 0.36.something, right?
Patches ran pretty clean with it, as they do when I use PD 0.37.1 extended from cvs with Gentoo on my AMD Shuttle box. It is just the P4 Linux laptop which makes my life less than ideal.
...but baby, maybe my luck will change tonight ;-) d.
yeah, on windows. i've definitely seen denormal cpu spikes here.
derek holzer wrote:
Josh Steiner wrote:
i have gotten denormal cpu spikes from the MSP version of pd. which version have you used that doesnt have them?
You mean for win, right?
Don't recall which versions I was using. Haven't booted windoze in over a year. But I am pretty sure it was PD extended, from Sourceforge, running on win2K, although exact version I am unsure of. If it was a year or so ago, that would mean 0.36.something, right?
Patches ran pretty clean with it, as they do when I use PD 0.37.1 extended from cvs with Gentoo on my AMD Shuttle box. It is just the P4 Linux laptop which makes my life less than ideal.
...but baby, maybe my luck will change tonight ;-) d.
well, there is pd's PD_BADFLOAT macro that _should_ to that, too, but this has only effect if the nubers are already denormal, so maybe it's a good idea to reset the value to flush every number a 24 / 32 bit dac would consider to be zero to zero...
Can you tell me a bit more about this? How and where should I start with that?
i changed the PD_BADFLOAT macro to detect small, almost denormal numbers (devel_0_37 branch). i had 3 evenings of several hours of performance without cpu spikes ... and i'd be interested if it works ok for you...
cheers...
Tim mailto:TimBlechmann@gmx.de ICQ: 96771783 -- The only people for me are the mad ones, the ones who are mad to live, mad to talk, mad to be saved, desirous of everything at the same time, the ones who never yawn or say a commonplace thing, but burn, burn, burn, like fabulous yellow roman candles exploding like spiders across the stars and in the middle you see the blue centerlight pop and everybody goes "Awww!" Jack Kerouac
-
derek holzer -
Josh Steiner -
Tim Blechmann