Hey gem devs,
How can I compiled gem without striping it?
I'm having trouble trying to use oprofile to profile the gem functions, it seems to just see the internal pd functions.
I tried removing the "strip" from the Makefile after doing autoconf/configure but that does not seem to be doing anything.
So I'm starting profile with:
opcontrol -i /path/to/pd --start
Stoping with:
opcontrol --shutdown
Looking at the results with:
opreport -d
How can I compile a Gem binary (on linux) so that I can profile with oprofile?
Thanks all. I'll think I'll be giving up soon, will be back at it again tommorrow, I really need to figure out what is causing these CPU jumps...
.b.
On Mon, May 15, 2006 1:17 pm, Tim Blechmann said:
Thanks tim, how do I compile an unstriped binary? (./configure --enable-debug?)
profiling with debugging might produce wrong results ... i guess you can
just remove the "strip" call from the makefile ...
t
-- TimBlechmann@gmx.de ICQ: 96771783 http://www.mokabar.tk
Desperation is the raw material of drastic change. Only those who can
leave behind everything they have ever believed in can hope to escape.
William S. Burroughs _______________________________________________ PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
B. Bogart wrote:
Hey gem devs,
How can I compiled gem without striping it?
when you have already run configure, edit Make.config and replace the STRIP-command with "echo"
I'm having trouble trying to use oprofile to profile the gem functions, it seems to just see the internal pd functions.
I tried removing the "strip" from the Makefile after doing autoconf/configure but that does not seem to be doing anything.
So I'm starting profile with:
opcontrol -i /path/to/pd --start
Stoping with:
opcontrol --shutdown
Looking at the results with:
opreport -d
How can I compile a Gem binary (on linux) so that I can profile with oprofile?
Thanks all. I'll think I'll be giving up soon, will be back at it again tommorrow, I really need to figure out what is causing these CPU jumps...
a very simple profiler for linux is qprof.
mfgadsr IOhannes
Hi Johannes,
When I do: autoconf && ./configure && vi Make.config
I only see the following lines containing "[sS][tT][rR][iI][pP]":
STRIP_UNNEEDED ?= --strip-unneeded STRIP ?= strip
So I'll comment those out and see what happens.
Thanks. .b.
On Tue, May 16, 2006 2:44 am, IOhannes m zmoelnig said:
B. Bogart wrote:
Hey gem devs,
How can I compiled gem without striping it?
when you have already run configure, edit Make.config and replace the STRIP-command with "echo"
I'm having trouble trying to use oprofile to profile the gem functions, it seems to just see the internal pd functions.
I tried removing the "strip" from the Makefile after doing autoconf/configure but that does not seem to be doing anything.
So I'm starting profile with:
opcontrol -i /path/to/pd --start
Stoping with:
opcontrol --shutdown
Looking at the results with:
opreport -d
How can I compile a Gem binary (on linux) so that I can profile with oprofile?
Thanks all. I'll think I'll be giving up soon, will be back at it again tommorrow, I really need to figure out what is causing these CPU jumps...
a very simple profiler for linux is qprof.
mfgadsr IOhannes
B. Bogart wrote:
Hi Johannes,
When I do: autoconf && ./configure && vi Make.config
I only see the following lines containing "[sS][tT][rR][iI][pP]":
STRIP_UNNEEDED ?= --strip-unneeded STRIP ?= strip
So I'll comment those out and see what happens.
uncommenting is a not-so-good idea.
try ~> STRIP=echo make without(!) editing the Make.config
(or alternatively change the line "STRIP ?= strip" to "STRIP ?= echo" -- actually that is what i meant by "replacing the STRIP-command with 'echo'")
mfg.asdr. IOhannes
Thanks Johannes,
So I changed "STRIP ?= strip" to "STRIP ?= echo"
oprofile still seems unable to see any gem symbols, it only shows me pd functions.
qprof is really useless for some reason:
% qprof -o profile -g function ~/pd-stuff/pd-0.38-4/bin/pd -open unified9.pd
patch runs for a while and then freezes (gemwin stops updating, and the CPU usage for PD drops to 0%.) qprof just sits there and I have to manually kill pd (control C).
the STDERR from qprof says:
% cat profile qprof: Ignoring sigaction(27, ...)
blech.
How can I test if my binary is striped or not? here is end of the gem build:
g++ -c -fPIC -g -O2 -freg-struct-return -O3 -fno-builtin -falign-loops=16 -funroll-loops -ffast-math -mmmx -I/usr/X11R6/include -I/usr/include/ffmpeg -I/usr/include/quicktime -I/usr/include/avifile-0.7 -I/usr/include/freetype2 -I.. -I/usr/include/FTGL -I/usr/include/freetype2 GEMglViewport.cpp -o GEMglViewport.o g++ -c -fPIC -g -O2 -freg-struct-return -O3 -fno-builtin -falign-loops=16 -funroll-loops -ffast-math -mmmx -I/usr/X11R6/include -I/usr/include/ffmpeg -I/usr/include/quicktime -I/usr/include/avifile-0.7 -I/usr/include/freetype2 -I.. -I/usr/include/FTGL -I/usr/include/freetype2 GEMgluPerspective.cpp -o GEMgluPerspective.o g++ -c -fPIC -g -O2 -freg-struct-return -O3 -fno-builtin -falign-loops=16 -funroll-loops -ffast-math -mmmx -I/usr/X11R6/include -I/usr/include/ffmpeg -I/usr/include/quicktime -I/usr/include/avifile-0.7 -I/usr/include/freetype2 -I.. -I/usr/include/FTGL -I/usr/include/freetype2 GLdefine.cpp -o GLdefine.o make[1]: Leaving directory `/home/bbogart/pd-stuff/Gem.current/src/openGL' g++ -o Gem.pd_linux -shared -Wl,-export-dynamic -L/usr/X11R6/lib Base/*.o Controls/*.o Geos/*.o Manips/*.o MarkEx/*.o Nongeos/*.o Particles/*.o Pixes/*.o openGL/*.o -ldv -lavformat -lmpeg -lmpeg3 -lquicktime -ljpeg -ltiff -lGLU -lGL -lXext -lXxf86vm -lXext -lX11 -ldl -lz -lm -L/usr/X11R6/lib -lpthread -lGLU -lGL -lfreetype -lz -lftgl -laviplay -L/usr/lib -lquicktime -lgthread -lz -ldl bbogart@insitu:~/pd-stuff/Gem.current/src$ sudo cp Gem.pd_linux /usr/lib/pd/extra/Gem.pd_linux + sudo cp Gem.pd_linux /usr/lib/pd/extra/Gem.pd_linu
Thanks, .b.
On Tue, May 16, 2006 9:27 am, IOhannes m zmoelnig said:
B. Bogart wrote:
Hi Johannes,
When I do: autoconf && ./configure && vi Make.config
I only see the following lines containing "[sS][tT][rR][iI][pP]":
STRIP_UNNEEDED ?= --strip-unneeded STRIP ?= strip
So I'll comment those out and see what happens.
uncommenting is a not-so-good idea.
try ~> STRIP=echo make without(!) editing the Make.config
(or alternatively change the line "STRIP ?= strip" to "STRIP ?= echo" -- actually that is what i meant by "replacing the STRIP-command with 'echo'")
mfg.asdr. IOhannes
Hey all,
we'll I've narrowed down the issue to lots of pix_buffer writing, but I can't be sure without getting the profiling to work. No matter what I do oprofile just shows me pd symbols, not the gem ones.
Anyhow the patch very simply moves the camera to look at different things in the room, if the pix_motion -> pix_blob analysis it decides if the frame is worth capturing. Then it gets added to a pix_buffer.
It cycles through 50 buffers each with 3 slots.
The CPU usage stays around 25% most of the time, going up to maybe 35%. Then once and a while (and more and more and longer and longer while the patch is running) it jumps up to 100%. Eventually the patch seems to stay at 100%. even if I stop the cycling, and therefore stop the capturing, the motion analsys and everything, all that is left is the video capture (v4l2 on winTV 640x480, usually uses 5-8%), and the CPU usage stays 100%. I've now left it for about 20min after stopping the cycle and the CPU usage is still 100%.
Could someone please help me profile this thing so I can at least find the cause?
I'm going to get something to eat now, so I'll be back on #dataflow in an hour.
Thanks all. .b.
On Tue, May 16, 2006 9:27 am, IOhannes m zmoelnig said:
B. Bogart wrote:
Hi Johannes,
When I do: autoconf && ./configure && vi Make.config
I only see the following lines containing "[sS][tT][rR][iI][pP]":
STRIP_UNNEEDED ?= --strip-unneeded STRIP ?= strip
So I'll comment those out and see what happens.
uncommenting is a not-so-good idea.
try ~> STRIP=echo make without(!) editing the Make.config
(or alternatively change the line "STRIP ?= strip" to "STRIP ?= echo" -- actually that is what i meant by "replacing the STRIP-command with 'echo'")
mfg.asdr. IOhannes
On 5/16/06, B. Bogart ben@ekran.org wrote:
Hey all,
we'll I've narrowed down the issue to lots of pix_buffer writing, but I can't be sure without getting the profiling to work. No matter what I do oprofile just shows me pd symbols, not the gem ones.
Did you try the AMD profiler? It is pretty easy to use and profiles all running processes on the machine. It will even read the source code and point out which lines of code took time in the sample. The GUI isn't Apple level of slick but it will show useful info.
Hi Chris,
I'm downloading "codeanalyst" TMTMTMTM now.
also googling on why oprofile may not be seeing my gem symbols.. (oprofile is also mentioned on the AMD site)
blechmann, help!
,b,
On Tue, May 16, 2006 12:30 pm, chris clepper said:
On 5/16/06, B. Bogart ben@ekran.org wrote:
Hey all,
we'll I've narrowed down the issue to lots of pix_buffer writing, but I can't be sure without getting the profiling to work. No matter what I do oprofile just shows me pd symbols, not the gem ones.
Did you try the AMD profiler? It is pretty easy to use and profiles all running processes on the machine. It will even read the source code and point out which lines of code took time in the sample. The GUI isn't Apple level of slick but it will show useful info. _______________________________________________ GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
Hi Johannes,
So I finally got oprofile to work (FYI its what the AMD Codeanalyst thing is based one, actually links to) with:
opcontrol --i pd -p library
with the non-striped pd binary.
So here is the results:
CPU: AMD64 processors, speed 2003.16 MHz (estimated) Counted CPU_CLK_UNHALTED events (Cycles outside of halt state) with a unit mask of 0x00 (No unit mask) count 100000 CPU_CLK_UNHALT...| samples| %| ------------------ 4693599 100.000 pd CPU_CLK_UNHALT...| samples| %| ------------------ 2671578 56.9196 libGLcore.so.1.0.7664 1129198 24.0583 Gem.pd_linux 256682 5.4688 libc-2.3.6.so 247330 5.2695 libfreetype.so.6.3.5 212402 4.5254 pd 127621 2.7190 libGL.so.1.0.7664 38593 0.8222 libX11.so.6.2 5956 0.1269 zero 2353 0.0501 libpthread-2.3.6.so 1093 0.0233 libGLU.so.1.3 793 0.0169 comport.pd_linux
So clearly its the nvidia driver sucking up all the CPU.
So I downloaded 8756 and I'm going to try that, though I have to recompile my kernel since I've gone up to gcc 4 since compiling the driver and kernel, and they both have to match...
So onto kernel compilation...
Johannes, what kernel/nvidia driver/gcc are you using?
I'll be at 2.6.16,gcc4,nvidia 8756 if I get this line of thought to work.
.b.
On Tue, May 16, 2006 1:28 pm, B. Bogart said:
Hi Chris,
I'm downloading "codeanalyst" TMTMTMTM now.
also googling on why oprofile may not be seeing my gem symbols.. (oprofile is also mentioned on the AMD site)
blechmann, help!
,b,
On Tue, May 16, 2006 12:30 pm, chris clepper said:
On 5/16/06, B. Bogart ben@ekran.org wrote:
Hey all,
we'll I've narrowed down the issue to lots of pix_buffer writing, but I can't be sure without getting the profiling to work. No matter what I do oprofile just shows me pd symbols, not the gem ones.
Did you try the AMD profiler? It is pretty easy to use and profiles all running processes on the machine. It will even read the source code and point out which lines of code took time in the sample. The GUI isn't Apple level of slick but it will show useful info. _______________________________________________ GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
Hey all,
So I'm going to leave the patch on overnight and *HOPE* that it does not due the weird spike while I'm gone.
Right now this is what the profile looks like:
5028091 51.5756 libGLcore.so.1.0.8756 2287523 23.4643 Gem.pd_linux 1155076 11.8482 zero 402984 4.1336 pd 336232 3.4489 libfreetype.so.6.3.5 210094 2.1550 libc-2.3.6.so 179207 1.8382 libm-2.3.6.so 75334 0.7727 libGL.so.1.0.8756 63900 0.6555 libX11.so.6.2 4586 0.0470 libpthread-2.3.6.so 3691 0.0379 libGLU.so.1.3 1618 0.0166 comport.pd_linux 199 0.0020 expr.pd_linux 182 0.0019 libpython2.3.so.1.0 78 8.0e-04 zexy.pd_linux 72 7.4e-04 libstdc++.so.6.0.8 46 4.7e-04 py.pd_linux 30 3.1e-04 xsample.pd_linux 17 1.7e-04 pool.pd_linux 11 1.1e-04 prepend.pd_linux
So the question is, what the heck is "zero" and why is pd linked with it???
I really hope this newer binary fixes the issue.
Thanks all for your help.
.b.
On Tue, May 16, 2006 2:12 pm, B. Bogart said:
Hi Johannes,
So I finally got oprofile to work (FYI its what the AMD Codeanalyst thing is based one, actually links to) with:
opcontrol --i pd -p library
with the non-striped pd binary.
So here is the results:
CPU: AMD64 processors, speed 2003.16 MHz (estimated) Counted CPU_CLK_UNHALTED events (Cycles outside of halt state) with a unit mask of 0x00 (No unit mask) count 100000 CPU_CLK_UNHALT...| samples| %|
4693599 100.000 pd CPU_CLK_UNHALT...| samples| %| ------------------ 2671578 56.9196 libGLcore.so.1.0.7664 1129198 24.0583 Gem.pd_linux 256682 5.4688 libc-2.3.6.so 247330 5.2695 libfreetype.so.6.3.5 212402 4.5254 pd 127621 2.7190 libGL.so.1.0.7664 38593 0.8222 libX11.so.6.2 5956 0.1269 zero 2353 0.0501 libpthread-2.3.6.so 1093 0.0233 libGLU.so.1.3 793 0.0169 comport.pd_linux
So clearly its the nvidia driver sucking up all the CPU.
So I downloaded 8756 and I'm going to try that, though I have to recompile my kernel since I've gone up to gcc 4 since compiling the driver and kernel, and they both have to match...
So onto kernel compilation...
Johannes, what kernel/nvidia driver/gcc are you using?
I'll be at 2.6.16,gcc4,nvidia 8756 if I get this line of thought to work.
.b.
On Tue, May 16, 2006 1:28 pm, B. Bogart said:
Hi Chris,
I'm downloading "codeanalyst" TMTMTMTM now.
also googling on why oprofile may not be seeing my gem symbols.. (oprofile is also mentioned on the AMD site)
blechmann, help!
,b,
On Tue, May 16, 2006 12:30 pm, chris clepper said:
On 5/16/06, B. Bogart ben@ekran.org wrote:
Hey all,
we'll I've narrowed down the issue to lots of pix_buffer writing, but I can't be sure without getting the profiling to work. No matter what I do oprofile just shows me pd symbols, not the gem ones.
Did you try the AMD profiler? It is pretty easy to use and profiles all running processes on the machine. It will even read the source code and point out which lines of code took time in the sample. The GUI isn't Apple level of slick but it will show useful info. _______________________________________________ GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
I think you need a better profiler which traces the calls. Just listing the libs is only marginally more useful than top. There's nothing to indicate if the driver is spending time uploading textures or vertices or just waiting for the GPU or doing something stupid. Without that info there is little anyone can do to fix any problems.
On 5/16/06, B. Bogart ben@ekran.org wrote:
Hey all,
So I'm going to leave the patch on overnight and *HOPE* that it does not due the weird spike while I'm gone.
Right now this is what the profile looks like:
5028091 51.5756 libGLcore.so.1.0.8756 2287523 23.4643 Gem.pd_linux 1155076 11.8482 zero 402984 4.1336 pd 336232 3.4489 libfreetype.so.6.3.5 210094 2.1550 libc-2.3.6.so 179207 1.8382 libm-2.3.6.so 75334 0.7727 libGL.so.1.0.8756 63900 0.6555 libX11.so.6.2 4586 0.0470 libpthread-2.3.6.so 3691 0.0379 libGLU.so.1.3 1618 0.0166 comport.pd_linux 199 0.0020 expr.pd_linux 182 0.0019 libpython2.3.so.1.0 78 8.0e-04 zexy.pd_linux 72 7.4e-04 libstdc++.so.6.0.8 46 4.7e-04 py.pd_linux 30 3.1e-04 xsample.pd_linux 17 1.7e-04 pool.pd_linux 11 1.1e-04 prepend.pd_linuxSo the question is, what the heck is "zero" and why is pd linked with it???
I really hope this newer binary fixes the issue.
Thanks all for your help.
Hallo!
I think you need a better profiler which traces the calls. Just listing the libs is only marginally more useful than top. There's nothing to
I didn't follow the whole thread, but did you try qprof ? It is really easy to use (qprof pd -lib Gem ...) and you get more information as you posted ...
LG Georg
On Tue, 2006-05-16 at 17:52 -0500, chris clepper wrote:
I think you need a better profiler which traces the calls. Just listing the libs is only marginally more useful than top.
asking for a better profiler is absolutely nonsense ... oprofile is perfectly able to trace calls, if the binary contains these information ...
http://oprofile.sourceforge.net/examples/
hth ... tim
-- TimBlechmann@gmx.de ICQ: 96771783 http://www.mokabar.tk
After one look at this planet any visitor from outer space would say "I want to see the manager." William S. Burroughs
Hi Chris,
I'm trying to do some google searching on how to debug the nvidia binary driver, but looks like (perhaps) only nvidia can do that...
Anyhow attached is the oprofile with the function calls, which does not tell me about anything in Gem I did not already know, and the two biggest calls, for rubber and the colour conversion, add up to 14%. 50% is sucked up by the black box of libGLCore.
The newer nvidia driver I installed yesterday does help, it takes longer for the CPU to get up to 100%, but certainly not not fix the problem.
If anyone knows: A: how to get more info about what the hell libGLcore is doing B: what the heck "zero" is and why its using 12% CPU
If I can't get anywhere on this today I guess I'll have to keep rebooting the machine each 2hrs or so, I will be baby sitting it anyhow, and it is only 2 days, but the next installation is going to be 3 days starting May 23rd.
Thanks, .b.
n Tue, May 16, 2006 6:52 pm, chris clepper said:
I think you need a better profiler which traces the calls. Just listing the libs is only marginally more useful than top. There's nothing to indicate if the driver is spending time uploading textures or vertices or just waiting for the GPU or doing something stupid. Without that info there is little anyone can do to fix any problems.
On 5/16/06, B. Bogart ben@ekran.org wrote:
Hey all,
So I'm going to leave the patch on overnight and *HOPE* that it does not due the weird spike while I'm gone.
Right now this is what the profile looks like:
5028091 51.5756 libGLcore.so.1.0.8756 2287523 23.4643 Gem.pd_linux 1155076 11.8482 zero 402984 4.1336 pd 336232 3.4489 libfreetype.so.6.3.5 210094 2.1550 libc-2.3.6.so 179207 1.8382 libm-2.3.6.so 75334 0.7727 libGL.so.1.0.8756 63900 0.6555 libX11.so.6.2 4586 0.0470 libpthread-2.3.6.so 3691 0.0379 libGLU.so.1.3 1618 0.0166 comport.pd_linux 199 0.0020 expr.pd_linux 182 0.0019 libpython2.3.so.1.0 78 8.0e-04 zexy.pd_linux 72 7.4e-04 libstdc++.so.6.0.8 46 4.7e-04 py.pd_linux 30 3.1e-04 xsample.pd_linux 17 1.7e-04 pool.pd_linux 11 1.1e-04 prepend.pd_linuxSo the question is, what the heck is "zero" and why is pd linked with it???
I really hope this newer binary fixes the issue.
Thanks all for your help.
GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
Sorry I forgot the attachments.
..b..
Anyone have an nvidia contact I could send this stuff to? I think its far over the head of the "knowledgebase forums".
On Wed, May 17, 2006 8:28 am, B. Bogart said:
Hi Chris,
I'm trying to do some google searching on how to debug the nvidia binary driver, but looks like (perhaps) only nvidia can do that...
Anyhow attached is the oprofile with the function calls, which does not tell me about anything in Gem I did not already know, and the two biggest calls, for rubber and the colour conversion, add up to 14%. 50% is sucked up by the black box of libGLCore.
The newer nvidia driver I installed yesterday does help, it takes longer for the CPU to get up to 100%, but certainly not not fix the problem.
If anyone knows: A: how to get more info about what the hell libGLcore is doing B: what the heck "zero" is and why its using 12% CPU
If I can't get anywhere on this today I guess I'll have to keep rebooting the machine each 2hrs or so, I will be baby sitting it anyhow, and it is only 2 days, but the next installation is going to be 3 days starting May 23rd.
Thanks, .b.
n Tue, May 16, 2006 6:52 pm, chris clepper said:
I think you need a better profiler which traces the calls. Just listing the libs is only marginally more useful than top. There's nothing to indicate if the driver is spending time uploading textures or vertices or just waiting for the GPU or doing something stupid. Without that info there is little anyone can do to fix any problems.
On 5/16/06, B. Bogart ben@ekran.org wrote:
Hey all,
So I'm going to leave the patch on overnight and *HOPE* that it does not due the weird spike while I'm gone.
Right now this is what the profile looks like:
5028091 51.5756 libGLcore.so.1.0.8756 2287523 23.4643 Gem.pd_linux 1155076 11.8482 zero 402984 4.1336 pd 336232 3.4489 libfreetype.so.6.3.5 210094 2.1550 libc-2.3.6.so 179207 1.8382 libm-2.3.6.so 75334 0.7727 libGL.so.1.0.8756 63900 0.6555 libX11.so.6.2 4586 0.0470 libpthread-2.3.6.so 3691 0.0379 libGLU.so.1.3 1618 0.0166 comport.pd_linux 199 0.0020 expr.pd_linux 182 0.0019 libpython2.3.so.1.0 78 8.0e-04 zexy.pd_linux 72 7.4e-04 libstdc++.so.6.0.8 46 4.7e-04 py.pd_linux 30 3.1e-04 xsample.pd_linux 17 1.7e-04 pool.pd_linux 11 1.1e-04 prepend.pd_linuxSo the question is, what the heck is "zero" and why is pd linked with it???
I really hope this newer binary fixes the issue.
Thanks all for your help.
GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
Try getting the call stack for Gem. This should trace the GL driver function back to the Gem call that triggered it. Following Tim's link above try opreport -cl --demangle=smart
Having something that does more that dump a billion lines of text to the console will help immensely in sorting out the profile.
On 5/17/06, B. Bogart ben@ekran.org wrote:
Hi Chris,
I'm trying to do some google searching on how to debug the nvidia binary driver, but looks like (perhaps) only nvidia can do that...
Anyhow attached is the oprofile with the function calls, which does not tell me about anything in Gem I did not already know, and the two biggest calls, for rubber and the colour conversion, add up to 14%. 50% is sucked up by the black box of libGLCore.
The newer nvidia driver I installed yesterday does help, it takes longer for the CPU to get up to 100%, but certainly not not fix the problem.
If anyone knows: A: how to get more info about what the hell libGLcore is doing B: what the heck "zero" is and why its using 12% CPU
If I can't get anywhere on this today I guess I'll have to keep rebooting the machine each 2hrs or so, I will be baby sitting it anyhow, and it is only 2 days, but the next installation is going to be 3 days starting May 23rd.
Thanks, .b.
n Tue, May 16, 2006 6:52 pm, chris clepper said:
I think you need a better profiler which traces the calls. Just listing the libs is only marginally more useful than top. There's nothing to
indicate
if the driver is spending time uploading textures or vertices or just waiting for the GPU or doing something stupid. Without that info there
is
little anyone can do to fix any problems.
On 5/16/06, B. Bogart ben@ekran.org wrote:
Hey all,
So I'm going to leave the patch on overnight and *HOPE* that it does
not
due the weird spike while I'm gone.
Right now this is what the profile looks like:
5028091 51.5756 libGLcore.so.1.0.8756 2287523 23.4643 Gem.pd_linux 1155076 11.8482 zero 402984 4.1336 pd 336232 3.4489 libfreetype.so.6.3.5 210094 2.1550 libc-2.3.6.so 179207 1.8382 libm-2.3.6.so 75334 0.7727 libGL.so.1.0.8756 63900 0.6555 libX11.so.6.2 4586 0.0470 libpthread-2.3.6.so 3691 0.0379 libGLU.so.1.3 1618 0.0166 comport.pd_linux 199 0.0020 expr.pd_linux 182 0.0019 libpython2.3.so.1.0 78 8.0e-04 zexy.pd_linux 72 7.4e-04 libstdc++.so.6.0.8 46 4.7e-04 py.pd_linux 30 3.1e-04 xsample.pd_linux 17 1.7e-04 pool.pd_linux 11 1.1e-04 prepend.pd_linuxSo the question is, what the heck is "zero" and why is pd linked with it???
I really hope this newer binary fixes the issue.
Thanks all for your help.
GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
Here you go chris,
It looks about the same to me though, the only gem functions I'm seeing using a lot of CPU are colour conversion (I guess for the bttv capture) and the rubber dynamics stuff.)
I don't understand how the CPU keeps increasing over time, since since the patch is locked in a cycle. The cycle does capture more and more frames over time into a buffer though. But if I stop the cycle so no new buffers are captures, the CPU stays at 100%.
The machine is now on a highspeed network so I can setup an account if anyone has the time and knowledge to help me with this.
I'm working on the rest of the stuff now, testing projectors, getting the touchscreen driver installed, trying some other nvidia drivers...
.b.
On Wed, May 17, 2006 8:42 am, chris clepper said:
Try getting the call stack for Gem. This should trace the GL driver function back to the Gem call that triggered it. Following Tim's link above try opreport -cl --demangle=smart
Having something that does more that dump a billion lines of text to the console will help immensely in sorting out the profile.
On 5/17/06, B. Bogart ben@ekran.org wrote:
Hi Chris,
I'm trying to do some google searching on how to debug the nvidia binary driver, but looks like (perhaps) only nvidia can do that...
Anyhow attached is the oprofile with the function calls, which does not tell me about anything in Gem I did not already know, and the two biggest calls, for rubber and the colour conversion, add up to 14%. 50% is sucked up by the black box of libGLCore.
The newer nvidia driver I installed yesterday does help, it takes longer for the CPU to get up to 100%, but certainly not not fix the problem.
If anyone knows: A: how to get more info about what the hell libGLcore is doing B: what the heck "zero" is and why its using 12% CPU
If I can't get anywhere on this today I guess I'll have to keep rebooting the machine each 2hrs or so, I will be baby sitting it anyhow, and it is only 2 days, but the next installation is going to be 3 days starting May 23rd.
Thanks, .b.
n Tue, May 16, 2006 6:52 pm, chris clepper said:
I think you need a better profiler which traces the calls. Just
listing
the libs is only marginally more useful than top. There's nothing to
indicate
if the driver is spending time uploading textures or vertices or just waiting for the GPU or doing something stupid. Without that info there
is
little anyone can do to fix any problems.
On 5/16/06, B. Bogart ben@ekran.org wrote:
Hey all,
So I'm going to leave the patch on overnight and *HOPE* that it does
not
due the weird spike while I'm gone.
Right now this is what the profile looks like:
5028091 51.5756 libGLcore.so.1.0.8756 2287523 23.4643 Gem.pd_linux 1155076 11.8482 zero 402984 4.1336 pd 336232 3.4489 libfreetype.so.6.3.5 210094 2.1550 libc-2.3.6.so 179207 1.8382 libm-2.3.6.so 75334 0.7727 libGL.so.1.0.8756 63900 0.6555 libX11.so.6.2 4586 0.0470 libpthread-2.3.6.so 3691 0.0379 libGLU.so.1.3 1618 0.0166 comport.pd_linux 199 0.0020 expr.pd_linux 182 0.0019 libpython2.3.so.1.0 78 8.0e-04 zexy.pd_linux 72 7.4e-04 libstdc++.so.6.0.8 46 4.7e-04 py.pd_linux 30 3.1e-04 xsample.pd_linux 17 1.7e-04 pool.pd_linux 11 1.1e-04 prepend.pd_linuxSo the question is, what the heck is "zero" and why is pd linked with it???
I really hope this newer binary fixes the issue.
Thanks all for your help.
GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
On 5/17/06, B. Bogart ben@ekran.org wrote:
Here you go chris,
It looks about the same to me though, the only gem functions I'm seeing using a lot of CPU are colour conversion (I guess for the bttv capture) and the rubber dynamics stuff.)
That is not all that helpful. Surely there is a way to get the real call tree. All of the time spent in libGLcore should be traced back to calls in GEM. That info is useful in your situation.
Can you just list the numbers of non-pix GEM objects you are using?
2 pix_texture 4 colorRGB etc
cgc
Hi Chris,
Here is the best count I can come up with (time permitting)
1 pix_video 6 gemheads 3 pix_texture 3 rectangle 1 pix_buffer_read 1 gemmouse 5 translate 3 rotate 3 scale 3 color 3 alpha 4 ortho 1 separator
20 instances of the same abstraction: (these are counts of the totals for the whole patch) 40 gemheads 100 pix_texture 120 rectangle 20 pix_buffer 60 pix_buffer_read 20 pix_buffer_write 160 translate 80 scale 20 color 40 alpha 160 separator
I really have no idea how the nv driver works, what your saying tells me that each gem GL function should execute a function in the nvidia driver. But since the driver is closed source without symbols would there be any way of getting those calls?
What I know for sure:
happens on the nvidia driver 7xxx and 8xxx.
The less copies of the abstraction I use the better is gets, that is the CPU increase takes longer, so it runs fine for longer. It also means gem captures less frames and therefore uses less memory.
It does not seem to happen with only 1 copy of the abstraction, but that probably just means that it would so be prolonged that i can't see the increase of CPU cycles.
attached is the patch, though it depends on pix_video uses in a weird way, and my serial control camera (and rubber from a recent CVS gem) load with -noloadbang first and remove what you know will not work. search for pix_video as well!
Thanks so much for keeping the dialog going. .b.
On Wed, May 17, 2006 10:37 am, chris clepper said:
On 5/17/06, B. Bogart ben@ekran.org wrote:
Here you go chris,
It looks about the same to me though, the only gem functions I'm seeing using a lot of CPU are colour conversion (I guess for the bttv capture) and the rubber dynamics stuff.)
That is not all that helpful. Surely there is a way to get the real call tree. All of the time spent in libGLcore should be traced back to calls in GEM. That info is useful in your situation.
Can you just list the numbers of non-pix GEM objects you are using?
2 pix_texture 4 colorRGB etc
cgc
On 5/17/06, B. Bogart ben@ekran.org wrote:
20 instances of the same abstraction: (these are counts of the totals for the whole patch) 40 gemheads 100 pix_texture 120 rectangle 20 pix_buffer 60 pix_buffer_read 20 pix_buffer_write 160 translate 80 scale 20 color 40 alpha 160 separator
That many separators will likely stall the GPU quite a bit since that basically resets the rendering pipeline. That many pix_texture objects constantly uploading textures will also stall in the driver on Linux and Windows (your patch would probably go like 0.2 fps on Windows).
I really have no idea how the nv driver works, what your saying tells me
that each gem GL function should execute a function in the nvidia driver. But since the driver is closed source without symbols would there be any way of getting those calls?
The profiler will take note of any memory location in the driver that takes time and then it should also keep track of what calls lead up to it. So if you do a glColor4f() in GEM that will make the driver do something with that call.
The symbols in the driver aren't all that important unless there is a bug to alert Nvidia about.
attached is the patch, though it depends on pix_video uses in a weird way,
and my serial control camera
A Pan/Tilt/Zoom camera?
cgc
Thanks chris,
in some cases it does seem possible to remove the separators, in others I suppose I would have to use matrixpop and matrixpush... anyone have an example patch of this?
so I removed many of the separators from my patch and we'll see if that improves things.
The patch seemed to be running fine before I put in [rubber], the abstraction that contains the rubber stuff is [single-moment]. But now that I think about it its probably just because the drop of frame-rate is only visible with rubber in it, maybe it was always doing this, but I don't recall so... I did upgrade gnome, but it seems very unlikely that would cause these troubles.
Yes its a pan/tilt/zoom camera. (but even if its the same static frame the patch should still act the same...)
I'm going to profile it now while its running fine.
Thanks again Chris, I appreciate your time. I'm sitting on #dataflow if you have any more questions.
.b.
On Wed, May 17, 2006 12:23 pm, chris clepper said:
On 5/17/06, B. Bogart ben@ekran.org wrote:
20 instances of the same abstraction: (these are counts of the totals for the whole patch) 40 gemheads 100 pix_texture 120 rectangle 20 pix_buffer 60 pix_buffer_read 20 pix_buffer_write 160 translate 80 scale 20 color 40 alpha 160 separator
That many separators will likely stall the GPU quite a bit since that
basically resets the rendering pipeline. That many pix_texture objects constantly uploading textures will also stall in the driver on Linux and Windows (your patch would probably go like 0.2 fps on Windows).
I really have no idea how the nv driver works, what your saying tells me
that each gem GL function should execute a function in the nvidia driver. But since the driver is closed source without symbols would there be
any
way of getting those calls?
The profiler will take note of any memory location in the driver that takes time and then it should also keep track of what calls lead up to it. So if you do a glColor4f() in GEM that will make the driver do something with
that
call.
The symbols in the driver aren't all that important unless there is a
bug
to alert Nvidia about.
attached is the patch, though it depends on pix_video uses in a weird
way,
and my serial control camera
A Pan/Tilt/Zoom camera?
cgc _______________________________________________ GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
What confuses me is the weird jumps, the patch sits there happy at 50% for an hour, then slowly it goes up, get a few 100% spikes, then blocks, then the whole machine is up to 100% constantly.
without rubber and w/ the reduced separators (taking out the text3d) the patch is using 10% CPU.
55% gem 20% pd 20% Glcore
I guess I'll try building it up slowly...
If anyone has a patch with the matrix pop/push "separator" replacement please send it along. Or will that behave as badly as [separator]?
arg.
.b.
On Wed, May 17, 2006 1:06 pm, B. Bogart said:
Thanks chris,
in some cases it does seem possible to remove the separators, in others I suppose I would have to use matrixpop and matrixpush... anyone have an example patch of this?
so I removed many of the separators from my patch and we'll see if that improves things.
The patch seemed to be running fine before I put in [rubber], the abstraction that contains the rubber stuff is [single-moment]. But now that I think about it its probably just because the drop of frame-rate is only visible with rubber in it, maybe it was always doing this, but I don't recall so... I did upgrade gnome, but it seems very unlikely that would cause these troubles.
Yes its a pan/tilt/zoom camera. (but even if its the same static frame the patch should still act the same...)
I'm going to profile it now while its running fine.
Thanks again Chris, I appreciate your time. I'm sitting on #dataflow if you have any more questions.
.b.
On Wed, May 17, 2006 12:23 pm, chris clepper said:
On 5/17/06, B. Bogart ben@ekran.org wrote:
20 instances of the same abstraction: (these are counts of the totals for the whole patch) 40 gemheads 100 pix_texture 120 rectangle 20 pix_buffer 60 pix_buffer_read 20 pix_buffer_write 160 translate 80 scale 20 color 40 alpha 160 separator
That many separators will likely stall the GPU quite a bit since that
basically resets the rendering pipeline. That many pix_texture objects constantly uploading textures will also stall in the driver on Linux and Windows (your patch would probably go like 0.2 fps on Windows).
I really have no idea how the nv driver works, what your saying tells me
that each gem GL function should execute a function in the nvidia driver. But since the driver is closed source without symbols would there be
any
way of getting those calls?
The profiler will take note of any memory location in the driver that takes time and then it should also keep track of what calls lead up to it. So if you do a glColor4f() in GEM that will make the driver do something with
that
call.
The symbols in the driver aren't all that important unless there is a
bug
to alert Nvidia about.
attached is the patch, though it depends on pix_video uses in a weird
way,
and my serial control camera
A Pan/Tilt/Zoom camera?
cgc _______________________________________________ GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
On 5/17/06, B. Bogart ben@ekran.org wrote:
If anyone has a patch with the matrix pop/push "separator" replacement please send it along. Or will that behave as badly as [separator]?
separator pushes and pops the gl matrix.
does pix_separator have the same issues as "separator"?
No, it just copies the pixels in memory from one buffer to another. That does take CPU time though, so don't use a ton of those for best performance.
hey all,
...sorry I've been silent lately: recovering from a bad sore throat/ cold! This has definitely been the year of being sick for me, so far...
On May 17, 2006, at 1:06 PM, B. Bogart wrote:
in some cases it does seem possible to remove the separators, in others I suppose I would have to use matrixpop and matrixpush... anyone have an example patch of this?
...first off, you'd need to know what matrix you're wanting to push/ pop, fr'example: [GLdefine]<-[GL_MODELVIEW< | [GEMglPushMatrix]
...then all you'd need to do is to [GEMglPushMatrix] before something that would change that matrix, and [GEMglPopMatrix] afterward... [separator] basically does this for three matrices at the same time, at the beginning of it's Gem::render(): GL_COLOR, GL_TEXTURE, & GL_MODELVIEW...these are then popped back in the postrender() call...
so I removed many of the separators from my patch and we'll see if that improves things.
The patch seemed to be running fine before I put in [rubber], the abstraction that contains the rubber stuff is [single-moment]. But now that I think about it its probably just because the drop of frame- rate is only visible with rubber in it, maybe it was always doing this, but I don't recall so... I did upgrade gnome, but it seems very unlikely that would cause these troubles.
...I think this could be due to the immediate mode drawing in [rubber]: I'm currently trying to switch it over to vertex_arrays/ vertex_buffer_objects, but it's not exactly straightforward (need to rewrite how the "masses" are stored)...
jamie
james tittle wrote:
hey all,
...then all you'd need to do is to [GEMglPushMatrix] before something that would change that matrix, and [GEMglPopMatrix] afterward...[separator] basically does this for three matrices at the same time, at the beginning of it's Gem::render(): GL_COLOR, GL_TEXTURE, & GL_MODELVIEW...these are then popped back in the postrender() call...
so i am not really convinced that exchanging [separator] by an abstraction that does the pushing/popping will help you much.
so I removed many of the separators from my patch and we'll see if that improves things.
yes, lots of them are just redundant. (e.g. you don't need to [separator] at the last outlet of a [t a ... a]
The patch seemed to be running fine before I put in [rubber], the abstraction that contains the rubber stuff is [single-moment]. But now
it might be that you are experiencing a denormalization problem with [rubber] (but unklikely if you are on AMD)
mfg.dr IOhannes
Hi Chris,
does pix_separator have the same issues as "separator"?
.b.
On Wed, May 17, 2006 12:23 pm, chris clepper said:
On 5/17/06, B. Bogart ben@ekran.org wrote:
20 instances of the same abstraction: (these are counts of the totals for the whole patch) 40 gemheads 100 pix_texture 120 rectangle 20 pix_buffer 60 pix_buffer_read 20 pix_buffer_write 160 translate 80 scale 20 color 40 alpha 160 separator
That many separators will likely stall the GPU quite a bit since that basically resets the rendering pipeline. That many pix_texture objects constantly uploading textures will also stall in the driver on Linux and Windows (your patch would probably go like 0.2 fps on Windows).
I really have no idea how the nv driver works, what your saying tells me
that each gem GL function should execute a function in the nvidia driver. But since the driver is closed source without symbols would there be any way of getting those calls?
The profiler will take note of any memory location in the driver that takes time and then it should also keep track of what calls lead up to it. So if you do a glColor4f() in GEM that will make the driver do something with that call.
The symbols in the driver aren't all that important unless there is a bug to alert Nvidia about.
attached is the patch, though it depends on pix_video uses in a weird way,
and my serial control camera
A Pan/Tilt/Zoom camera?
cgc _______________________________________________ GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
B. Bogart wrote:
Hi Chris,
does pix_separator have the same issues as "separator"?
these are 2 completely different objects, so they don't share issues. ([separator] pushes/pops the OpenGL-state while [pix_separator] pushes/pops the pixel-buffer)
they do not interfere. you cannot exchange one for the other.
mfga.sdr. IOhannes
On Wed, 17 May 2006, B. Bogart wrote:
B: what the heck "zero" is and why its using 12% CPU
The names you see below are most likely the names of memory segments, which are normally filenames. One way to allocate memory is to mmap the "/dev/zero" file in MAP_PRIVATE mode. If the program-counter is caught sitting in such an area, it means that there might be something like runtime-generated code being run, or that some drivers are loaded in an unusual way, e.g. read() the executable into some malloc()ed or mmap()ed area...
note that malloc may use mmap, and in fact does, for blocks of 128k and more.
also note that since a few years, XFree86 and XOrg do not use the OS's dynamic-loader, in order to support more non-UNIX systems. (well that's what I heard). So I suspect that "zero" is the non-GL portion of the nVidia driver.
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju | Freelance Digital Arts Engineer, Montréal QC Canada
-
B. Bogart -
chris clepper -
Georg Holzmann -
IOhannes m zmoelnig -
james tittle -
Mathieu Bouchard -
Tim Blechmann