Hi list,
I'm developing a patch in PD/Gem on Linux where I'll have to play many hi-res videos at the same time. I did some research to find the best codec with the lowest CPU usage on Linux. I found that there's not a lot of well optimize codecs on Linux. Patrick via Ben B. told me that after a lot of testing he found that MPEG2 was the best choice. After some testing I found mostly the same thing: MPEG2 or MPEG4 are the most optimize codecs on Linux. I did also some testing with different libraries for playback. LibMPEG2 (MPlayer is using that one for MPEG2) is the fastest one for MPEG2 follow by FFMPEG. In my case I'll only play the file from start at normal speed, so it's not a problem if the codec have temporal compression. Just to give you an idea, I can play a 800x440 px, 24fps at a really good quality and using less then 15% of CPU on an P4 2,6GHz in MPEG2 with LibMPEG2 !
So here's my questions. I tried to compile Gem with FFMPEG support and it always fail in one manner or another. Or it doesn't build at all, or it build but when I load Gem I got this error message (undefined symbol: ogg_stream_init Gem: can't load library), or everything seems OK but when I try to load a movie, any type, in pix_film, FFMPEG always say that it doesn't support this format. So I can't play MPEG2 or MPEG4 in Gem, except with other less (very less) efficient libraries (it take around 50% CPU to play the same file with LibMPEG3). I did my compilation test on Unbuntu (Debian) and I try with the distro installation of FFMPEG and also with a version I compile myself (this one is more efficient in CPU usage). I tried with both Gem 0.90 and the CVS version. Same result. Could someone help me compiling Gem with a working FFMPEG ? I'm using PD 0.37
Also, what I would really prefer is to had support in Gem for LibMPEG2. Do you think it could be difficult to do ? Is it about creating a filmMPEG2.cpp and filmMPEG2.h files with the good function calls to the library and add the reference into the pix_filmNEW.cpp ?
And I would like to know what's the best container (.mov, .avi, .mpg, raw) to playback video in Gem.
One last thing, I was not able to use FSAA. My video card is an ATI 9600 and I'm using the binary driver with x.org. I put the lines to enable FSAA in my X86-config-4 file without any success. Any ideas ?
Thanks and sorry to ask so much questions in one time.
Etienne
On Wed, 1 Jun 2005, Etienne Desautels wrote:
Could someone help me compiling Gem with a working FFMPEG ? I'm using PD 0.37
I am into recompiling Gem on different platforms these days, so I can take a look at that. It is probably easier for us if I produce a deb package and you test the compiled version ?
Guenter
Etienne Desautels wrote:
Hi list,
So here's my questions. I tried to compile Gem with FFMPEG support and it always fail in one manner or another. Or it doesn't build at all, or it build but when I load Gem I got this error message (undefined symbol: ogg_stream_init Gem: can't load library), or everything seems OK but
hmm, are you using the CVS version of Gem ? there are some better linking options in the CVS-version (basically it adds the output of "ffmpeg-config --plugin-libs" which adds more libraries than just libavcodec and libavformat) if not, you will probably have to add these flags by hand: "-lavformat_pic -lavcodec_pic -lvorbis -lvorbisenc -ldts_pic -la52"
Also, what I would really prefer is to had support in Gem for LibMPEG2. Do you think it could be difficult to do ? Is it about creating a filmMPEG2.cpp and filmMPEG2.h files with the good function calls to the library and add the reference into the pix_filmNEW.cpp ?
correct. if you like to add libMPEG2-support you are welcome!
One last thing, I was not able to use FSAA. My video card is an ATI 9600 and I'm using the binary driver with x.org. I put the lines to enable FSAA in my X86-config-4 file without any success. Any ideas ?
sorry, no experience with Xorg
mfg.as.dr IOhannes
IOhannes and Günter, thanks for your fast answer, as usual.
This afternoon I'll try to recompile Gem with the advice of IOhannes. If it still doesn't work, I'll be glad to test your deb package Güenter. I'll keep you inform of my findings.
On 05-06-02, at 07:34, IOhannes m zmoelnig wrote:
Etienne Desautels wrote:
Hi list, So here's my questions. I tried to compile Gem with FFMPEG support and it always fail in one manner or another. Or it doesn't build at all, or it build but when I load Gem I got this error message (undefined symbol: ogg_stream_init Gem: can't load library), or everything seems OK but
hmm, are you using the CVS version of Gem ? there are some better linking options in the CVS-version (basically it adds the output of "ffmpeg-config --plugin-libs" which adds more libraries than just libavcodec and libavformat) if not, you will probably have to add these flags by hand: "-lavformat_pic -lavcodec_pic -lvorbis -lvorbisenc -ldts_pic -la52"
Also, what I would really prefer is to had support in Gem for LibMPEG2. Do you think it could be difficult to do ? Is it about creating a filmMPEG2.cpp and filmMPEG2.h files with the good function calls to the library and add the reference into the pix_filmNEW.cpp ?
correct. if you like to add libMPEG2-support you are welcome!
I would like to test the performance of FFMPEG in Gem before and if there's not a big difference I'll use FFMPEG. But if I didn't get satisfaction with FFMPEG, I will had the support for LibMPEG2 as it doesn't look to complicated to do.
One last thing, I was not able to use FSAA. My video card is an ATI 9600 and I'm using the binary driver with x.org. I put the lines to enable FSAA in my X86-config-4 file without any success. Any ideas ?
sorry, no experience with Xorg
No problem.
mfg.as.dr IOhannes
Etienne
Hi IOhannes, Günter,
here is the results of my tests to have a working version of Gem with FFMPEG support. I compile the CVS version of Gem with different versions of FFMPEG (distro, CVS, etc..., ffmpeg-config was there) and now I always get the same results: all is compiling fine and working fine except that every time I try to open a movie (that works without any problems in ffplay) in pix_film, FFMPEG is unable to open it and return err = -2. I check in the source code of FFMPEG to find the meaning of this error and it's an I/O error. I did a strace but it's over my comprehension. It look likes FFMPEG is trying to open the file in different ways without success. Any ideas ? Do somebody have a version of Gem with a working FFMPEG ?
Thanks
Etienne
On 05-06-02, at 07:34, IOhannes m zmoelnig wrote:
Etienne Desautels wrote:
Hi list, So here's my questions. I tried to compile Gem with FFMPEG support and it always fail in one manner or another. Or it doesn't build at all, or it build but when I load Gem I got this error message (undefined symbol: ogg_stream_init Gem: can't load library), or everything seems OK but
hmm, are you using the CVS version of Gem ? there are some better linking options in the CVS-version (basically it adds the output of "ffmpeg-config --plugin-libs" which adds more libraries than just libavcodec and libavformat) if not, you will probably have to add these flags by hand: "-lavformat_pic -lavcodec_pic -lvorbis -lvorbisenc -ldts_pic -la52"
Also, what I would really prefer is to had support in Gem for LibMPEG2. Do you think it could be difficult to do ? Is it about creating a filmMPEG2.cpp and filmMPEG2.h files with the good function calls to the library and add the reference into the pix_filmNEW.cpp ?
correct. if you like to add libMPEG2-support you are welcome!
One last thing, I was not able to use FSAA. My video card is an ATI 9600 and I'm using the binary driver with x.org. I put the lines to enable FSAA in my X86-config-4 file without any success. Any ideas ?
sorry, no experience with Xorg
mfg.as.dr IOhannes
hi all ...
Also, what I would really prefer is to had support in Gem for LibMPEG2. Do you think it could be difficult to do ? Is it about creating a filmMPEG2.cpp and filmMPEG2.h files with the good function calls to the library and add the reference into the pix_filmNEW.cpp ?
correct. if you like to add libMPEG2-support you are welcome!
stupid question, but what about mplayer support instead of libMPEG2? mplayer is able to play back about every file format ...
just thinking loud ... tim
Hello,
On 05-06-05, at 17:14, Tim Blechmann wrote:
hi all ...
Also, what I would really prefer is to had support in Gem for LibMPEG2. Do you think it could be difficult to do ? Is it about creating a filmMPEG2.cpp and filmMPEG2.h files with the good function calls to the library and add the reference into the pix_filmNEW.cpp ?
correct. if you like to add libMPEG2-support you are welcome!
stupid question, but what about mplayer support instead of libMPEG2? mplayer is able to play back about every file format ...
just thinking loud ... tim
I know, it would be wonderfull...
But Mplayer is not a library. You can only use it as an application. Mplayer itself use LibMPEG2 to play MPEG2 stream and use other libraries for other formats and codecs (as FFMPEG, XVID, etc).
Unfortunately, I will not add LibMPEG2 to GEM. In fact I'll not use PD/GEM at all for the big project I'm working on. Maybe I'm crazy but the problem is that PD is not multithreaded and work with CPU time and not real time. So if you try in Gem to do something as simple as rotating a square on the screen, it will not rotate regularly. It will stall from time to time. And if you try to load a big texture (1024x1024) the rotation of the square will stall even more badly. And I don't think there's something I can do ? I tried to preload some big textures with pix_buffer but when I display the image it's sucking a lot of CPU. When I say a lot I mean a lot. The more your texture is big the more CPU it's suck. It's very strange. I don't understand why? Anyway, in the mid-term it's not an option for me to preload the images. So now I'm doing the project in Python with pyOpenGL. This way I can schedule redraw in real time, not cpu time, and open my textures in a thread. This way everything play smoothly. But I have a lot more to code ! I wish I could done it with PD/GEM but it's not possible.
One last note: for FFMEPG in Gem, I did try so many things but in the end it never worked. I always have the -2 error meaning that there's an I/O problem?
Anyway thanks for your good work to all the developers of PD and Gem !
Etienne
Etienne,
Are these dropped frames your seeing a result of rotating the object via numberbox/PD GUI or from another machine running over -nogui? I ask because the PD gui is known to have issues and it is easily possible that it is the number box causing this problem and not Gem. Try sending a signal from python over a local socket to pd/Gem running in -nogui. Does it do the same thing?
I'll have to test things on my new linux machine when I get it, but I hope what your saying is not evident. I don't see this happening on OSX. Do you have a test patch?
Disappointed B.
:(
Etienne Desautels wrote:
Hello,
On 05-06-05, at 17:14, Tim Blechmann wrote:
hi all ...
Also, what I would really prefer is to had support in Gem for LibMPEG2. Do you think it could be difficult to do ? Is it about creating a filmMPEG2.cpp and filmMPEG2.h files with the good function calls to the library and add the reference into the pix_filmNEW.cpp ?
correct. if you like to add libMPEG2-support you are welcome!
stupid question, but what about mplayer support instead of libMPEG2? mplayer is able to play back about every file format ...
just thinking loud ... tim
I know, it would be wonderfull...
But Mplayer is not a library. You can only use it as an application. Mplayer itself use LibMPEG2 to play MPEG2 stream and use other libraries for other formats and codecs (as FFMPEG, XVID, etc).
Unfortunately, I will not add LibMPEG2 to GEM. In fact I'll not use PD/GEM at all for the big project I'm working on. Maybe I'm crazy but the problem is that PD is not multithreaded and work with CPU time and not real time. So if you try in Gem to do something as simple as rotating a square on the screen, it will not rotate regularly. It will stall from time to time. And if you try to load a big texture (1024x1024) the rotation of the square will stall even more badly. And I don't think there's something I can do ? I tried to preload some big textures with pix_buffer but when I display the image it's sucking a lot of CPU. When I say a lot I mean a lot. The more your texture is big the more CPU it's suck. It's very strange. I don't understand why? Anyway, in the mid-term it's not an option for me to preload the images. So now I'm doing the project in Python with pyOpenGL. This way I can schedule redraw in real time, not cpu time, and open my textures in a thread. This way everything play smoothly. But I have a lot more to code ! I wish I could done it with PD/GEM but it's not possible.
One last note: for FFMEPG in Gem, I did try so many things but in the end it never worked. I always have the -2 error meaning that there's an I/O problem?
Anyway thanks for your good work to all the developers of PD and Gem !
Etienne
Hey Ben,
Are these dropped frames your seeing a result of rotating the object via numberbox/PD GUI or from another machine running over -nogui?
No, either. Rotating from metro. (See the patch) I have the same problem on my 667 PowerBook and my Linux pentium 2.6GHz.
I ask because the PD gui is known to have issues and it is easily possible that it is the number box causing this problem and not Gem. Try sending a signal from python over a local socket to pd/Gem running in -nogui.
It will probably work. The problem is the internal timing of PD/metro. It's CPU time base, not real time base. And without threads.
Does it do the same thing?
I'll have to test things on my new linux machine when I get it, but I hope what your saying is not evident. I don't see this happening on OSX. Do you have a test patch?
Here it is !
Disappointed B.
Disappointed E. too.
#N canvas 513 371 819 325 10; #X obj 187 150 gemhead; #X msg 71 9 destroy; #X obj 187 129 tgl 15 0 empty empty empty 0 -6 0 8 -262144 -1 -1 0 1; #X obj 485 157 gemhead; #X obj 414 177 bng 15 250 50 0 empty empty empty 0 -6 0 8 -262144 -1 -1; #X obj 485 264 pix_texture; #X obj 485 287 rectangle 5.333 4; #X msg 414 216 open $1; #X obj 414 195 openpanel; #X obj 485 137 tgl 15 0 empty empty empty 0 -6 0 8 -262144 -1 -1 0 1; #X obj 485 240 pix_image; #X msg 503 186 open /home/etienne/Desktop/froime400.jpg; #X obj 187 198 rotateXYZ; #X obj 289 62 tgl 15 0 empty empty empty 0 -6 0 8 -262144 -1 -1 0 1 ; #X obj 340 62 nbx 5 14 -1e+37 1e+37 0 0 empty empty empty 0 -6 0 10 -262144 -1 -1 0 256; #X obj 289 127 + 1; #X obj 289 105 f 1; #X obj 187 221 rectangle 2 2; #X obj 327 106 nbx 5 14 -1e+37 1e+37 0 0 empty empty empty 0 -6 0 10 -262144 -1 -1 0 256; #X msg 503 210 open /home/etienne/Desktop/thuangS.jpg; #X msg 16 9 create; #X msg 36 30 1; #X msg 98 31 0; #X msg 91 86 frame $1; #X obj 16 113 gemwin 30; #X msg 91 60 30; #X msg 125 60 60; #X obj 289 82 metro 33; #X obj 485 89 loadbang; #X msg 485 113 0; #X text 130 9 1 create the window; #X text 233 29 2 start rotating the square watch it rotating not smoothly! ; #X msg 254 151 0; #X obj 289 151 select 361; #X text 483 48 3 try to load some big textures (1024x1024) with the chain turn on or off watch the square stall; #X connect 0 0 12 0; #X connect 1 0 24 0; #X connect 2 0 0 0; #X connect 3 0 10 0; #X connect 4 0 8 0; #X connect 5 0 6 0; #X connect 7 0 10 0; #X connect 8 0 7 0; #X connect 9 0 3 0; #X connect 10 0 5 0; #X connect 11 0 10 0; #X connect 12 0 17 0; #X connect 13 0 27 0; #X connect 14 0 27 1; #X connect 15 0 33 0; #X connect 16 0 15 0; #X connect 18 0 15 1; #X connect 19 0 10 0; #X connect 20 0 24 0; #X connect 21 0 24 0; #X connect 22 0 24 0; #X connect 23 0 24 0; #X connect 25 0 23 0; #X connect 26 0 23 0; #X connect 27 0 16 0; #X connect 28 0 29 0; #X connect 29 0 9 0; #X connect 32 0 16 0; #X connect 33 0 32 0; #X connect 33 1 16 1; #X connect 33 1 12 3;
:(
Etienne Desautels wrote:
Hello,
On 05-06-05, at 17:14, Tim Blechmann wrote:
hi all ...
Also, what I would really prefer is to had support in Gem for LibMPEG2. Do you think it could be difficult to do ? Is it about creating a filmMPEG2.cpp and filmMPEG2.h files with the good function calls to the library and add the reference into the pix_filmNEW.cpp ?
correct. if you like to add libMPEG2-support you are welcome!
stupid question, but what about mplayer support instead of libMPEG2? mplayer is able to play back about every file format ...
just thinking loud ... tim
I know, it would be wonderfull...
But Mplayer is not a library. You can only use it as an application. Mplayer itself use LibMPEG2 to play MPEG2 stream and use other libraries for other formats and codecs (as FFMPEG, XVID, etc).
Unfortunately, I will not add LibMPEG2 to GEM. In fact I'll not use PD/GEM at all for the big project I'm working on. Maybe I'm crazy but the problem is that PD is not multithreaded and work with CPU time and not real time. So if you try in Gem to do something as simple as rotating a square on the screen, it will not rotate regularly. It will stall from time to time. And if you try to load a big texture (1024x1024) the rotation of the square will stall even more badly. And I don't think there's something I can do ? I tried to preload some big textures with pix_buffer but when I display the image it's sucking a lot of CPU. When I say a lot I mean a lot. The more your texture is big the more CPU it's suck. It's very strange. I don't understand why? Anyway, in the mid-term it's not an option for me to preload the images. So now I'm doing the project in Python with pyOpenGL. This way I can schedule redraw in real time, not cpu time, and open my textures in a thread. This way everything play smoothly. But I have a lot more to code ! I wish I could done it with PD/GEM but it's not possible.
One last note: for FFMEPG in Gem, I did try so many things but in the end it never worked. I always have the -2 error meaning that there's an I/O problem?
Anyway thanks for your good work to all the developers of PD and Gem !
Etienne
Hi Etienne,
I'm trying your patch on my g4 now.
The square does rotate smoothly, unless I move the PD windows around.
loading an image DOES stall rendering, I thought you meant that the image having been loaded already slows down rendering, which this patch does not show.
So reading a big file from disk in pix_image does slow things. I did a test using pix_film in place of pix_image on OSX and that still stalls. Gem only stalls while loading, not once the texture IS loaded (on my setup). I wish I had my new linux machine to test...
So I guess you are proposing that the part that opens files be threaded so that loading the file off disk does not stall rendering?
This is impossible for your project because there is not enough RAM to hold all the textures needed?
I have a patch that loads about 20 JPG's at 512x512 plus 8 TIFs at 500KB each, on OSX with only 512MB of Ram and this works fine... all using pix_multiimage.
I've attached a change to your patch that uses the gemchain render signal as the rotation clock, rather than the metro, Does this also rotate choppily?
B.
Etienne Desautels wrote:
Hey Ben,
Are these dropped frames your seeing a result of rotating the object via numberbox/PD GUI or from another machine running over -nogui?
No, either. Rotating from metro. (See the patch) I have the same problem on my 667 PowerBook and my Linux pentium 2.6GHz.
I ask because the PD gui is known to have issues and it is easily possible that it is the number box causing this problem and not Gem. Try sending a signal from python over a local socket to pd/Gem running in -nogui.
It will probably work. The problem is the internal timing of PD/metro. It's CPU time base, not real time base. And without threads.
Does it do the same thing?
I'll have to test things on my new linux machine when I get it, but I hope what your saying is not evident. I don't see this happening on OSX. Do you have a test patch?
Here it is !
Disappointed B.
Disappointed E. too.
#N canvas 513 371 819 325 10; #X obj 187 150 gemhead; #X msg 71 9 destroy; #X obj 187 129 tgl 15 0 empty empty empty 0 -6 0 8 -262144 -1 -1 0 1; #X obj 485 157 gemhead; #X obj 414 177 bng 15 250 50 0 empty empty empty 0 -6 0 8 -262144 -1 -1; #X obj 485 264 pix_texture; #X obj 485 287 rectangle 5.333 4; #X msg 414 216 open $1; #X obj 414 195 openpanel; #X obj 485 137 tgl 15 0 empty empty empty 0 -6 0 8 -262144 -1 -1 0 1; #X obj 485 240 pix_image; #X msg 503 186 open /home/etienne/Desktop/froime400.jpg; #X obj 187 198 rotateXYZ; #X obj 289 62 tgl 15 0 empty empty empty 0 -6 0 8 -262144 -1 -1 0 1 ; #X obj 340 62 nbx 5 14 -1e+37 1e+37 0 0 empty empty empty 0 -6 0 10 -262144 -1 -1 0 256; #X obj 289 127 + 1; #X obj 289 105 f 1; #X obj 187 221 rectangle 2 2; #X obj 327 106 nbx 5 14 -1e+37 1e+37 0 0 empty empty empty 0 -6 0 10 -262144 -1 -1 0 256; #X msg 503 210 open /home/etienne/Desktop/thuangS.jpg; #X msg 16 9 create; #X msg 36 30 1; #X msg 98 31 0; #X msg 91 86 frame $1; #X obj 16 113 gemwin 30; #X msg 91 60 30; #X msg 125 60 60; #X obj 289 82 metro 33; #X obj 485 89 loadbang; #X msg 485 113 0; #X text 130 9 1 create the window; #X text 233 29 2 start rotating the square watch it rotating not smoothly! ; #X msg 254 151 0; #X obj 289 151 select 361; #X text 483 48 3 try to load some big textures (1024x1024) with the chain turn on or off watch the square stall; #X connect 0 0 12 0; #X connect 1 0 24 0; #X connect 2 0 0 0; #X connect 3 0 10 0; #X connect 4 0 8 0; #X connect 5 0 6 0; #X connect 7 0 10 0; #X connect 8 0 7 0; #X connect 9 0 3 0; #X connect 10 0 5 0; #X connect 11 0 10 0; #X connect 12 0 17 0; #X connect 13 0 27 0; #X connect 14 0 27 1; #X connect 15 0 33 0; #X connect 16 0 15 0; #X connect 18 0 15 1; #X connect 19 0 10 0; #X connect 20 0 24 0; #X connect 21 0 24 0; #X connect 22 0 24 0; #X connect 23 0 24 0; #X connect 25 0 23 0; #X connect 26 0 23 0; #X connect 27 0 16 0; #X connect 28 0 29 0; #X connect 29 0 9 0; #X connect 32 0 16 0; #X connect 33 0 32 0; #X connect 33 1 16 1; #X connect 33 1 12 3;
:(
Etienne Desautels wrote:
Hello,
On 05-06-05, at 17:14, Tim Blechmann wrote:
hi all ...
Also, what I would really prefer is to had support in Gem for LibMPEG2. Do you think it could be difficult to do ? Is it about creating a filmMPEG2.cpp and filmMPEG2.h files with the good function calls to the library and add the reference into the pix_filmNEW.cpp ?
correct. if you like to add libMPEG2-support you are welcome!
stupid question, but what about mplayer support instead of libMPEG2? mplayer is able to play back about every file format ...
just thinking loud ... tim
I know, it would be wonderfull...
But Mplayer is not a library. You can only use it as an application. Mplayer itself use LibMPEG2 to play MPEG2 stream and use other libraries for other formats and codecs (as FFMPEG, XVID, etc).
Unfortunately, I will not add LibMPEG2 to GEM. In fact I'll not use PD/GEM at all for the big project I'm working on. Maybe I'm crazy but the problem is that PD is not multithreaded and work with CPU time and not real time. So if you try in Gem to do something as simple as rotating a square on the screen, it will not rotate regularly. It will stall from time to time. And if you try to load a big texture (1024x1024) the rotation of the square will stall even more badly. And I don't think there's something I can do ? I tried to preload some big textures with pix_buffer but when I display the image it's sucking a lot of CPU. When I say a lot I mean a lot. The more your texture is big the more CPU it's suck. It's very strange. I don't understand why? Anyway, in the mid-term it's not an option for me to preload the images. So now I'm doing the project in Python with pyOpenGL. This way I can schedule redraw in real time, not cpu time, and open my textures in a thread. This way everything play smoothly. But I have a lot more to code ! I wish I could done it with PD/GEM but it's not possible.
One last note: for FFMEPG in Gem, I did try so many things but in the end it never worked. I always have the -2 error meaning that there's an I/O problem?
Anyway thanks for your good work to all the developers of PD and Gem !
Etienne
GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
#N canvas 360 448 907 335 10; #X obj 187 80 gemhead; #X msg 71 9 destroy; #X obj 187 59 tgl 15 0 empty empty empty 0 -6 0 8 -262144 -1 -1 0 1 ; #X obj 485 157 gemhead; #X obj 414 177 bng 15 250 50 0 empty empty empty 0 -6 0 8 -262144 -1 -1; #X obj 485 264 pix_texture; #X obj 485 287 rectangle 5.333 4; #X msg 414 216 open $1; #X obj 414 195 openpanel; #X obj 485 137 tgl 15 0 empty empty empty 0 -6 0 8 -262144 -1 -1 0 1; #X msg 503 186 open /home/etienne/Desktop/froime400.jpg; #X obj 179 258 rotateXYZ; #X obj 274 73 tgl 15 0 empty empty empty 0 -6 0 8 -262144 -1 -1 0 1 ; #X obj 325 73 nbx 5 14 -1e+37 1e+37 0 0 empty empty empty 0 -6 0 10 -262144 -1 -1 0 256; #X obj 237 185 + 1; #X obj 237 163 f 1; #X obj 179 281 rectangle 2 2; #X obj 275 164 nbx 5 14 -1e+37 1e+37 0 0 empty empty empty 0 -6 0 10 -262144 -1 -1 0 256; #X msg 503 210 open /home/etienne/Desktop/thuangS.jpg; #X msg 16 9 create; #X msg 36 30 1; #X msg 98 31 0; #X msg 91 86 frame $1; #X obj 16 113 gemwin 30; #X msg 91 60 30; #X msg 125 60 60; #X obj 274 93 metro 33; #X obj 485 89 loadbang; #X msg 485 113 0; #X text 130 9 1 create the window; #X text 233 29 2 start rotating the square watch it rotating not smoothly! ; #X msg 202 209 0; #X obj 237 209 select 361; #X text 483 48 3 try to load some big textures (1024x1024) with the chain turn on or off watch the square stall; #X obj 485 240 pix_film; #X obj 187 116 t a b; #X text 89 119 Ben magic ->; #X connect 0 0 35 0; #X connect 1 0 23 0; #X connect 2 0 0 0; #X connect 3 0 34 0; #X connect 4 0 8 0; #X connect 5 0 6 0; #X connect 7 0 34 0; #X connect 8 0 7 0; #X connect 9 0 3 0; #X connect 10 0 34 0; #X connect 11 0 16 0; #X connect 12 0 26 0; #X connect 13 0 26 1; #X connect 14 0 32 0; #X connect 15 0 14 0; #X connect 17 0 14 1; #X connect 18 0 34 0; #X connect 19 0 23 0; #X connect 20 0 23 0; #X connect 21 0 23 0; #X connect 22 0 23 0; #X connect 24 0 22 0; #X connect 25 0 22 0; #X connect 27 0 28 0; #X connect 28 0 9 0; #X connect 31 0 15 0; #X connect 32 0 31 0; #X connect 32 1 15 1; #X connect 32 1 11 3; #X connect 34 0 5 0; #X connect 35 0 11 0; #X connect 35 1 15 0;
Hi Ben,
I'm trying your patch on my g4 now.
The square does rotate smoothly, unless I move the PD windows around.
On mine, (I know, it's pretty old!) it never rotate smoothly. I don't have to move windows to see the effect. It doesn't stall like when you load an image, it's more subtle then that but it's too much for me and for the project. And I have the same results on my Linux box.
loading an image DOES stall rendering, I thought you meant that the image having been loaded already slows down rendering, which this patch does not show.
Exactly, no problem with already loaded images, just when you load images.
So reading a big file from disk in pix_image does slow things. I did a test using pix_film in place of pix_image on OSX and that still stalls.
Yes.
Gem only stalls while loading, not once the texture IS loaded (on my setup).
Yes that's true also for me.
I wish I had my new linux machine to test...
So I guess you are proposing that the part that opens files be threaded so that loading the file off disk does not stall rendering?
Exactly. This is what I'm doing in python.
This is impossible for your project because there is not enough RAM to hold all the textures needed?
Yes it's impossible. The minimum acceptable size and number of images I have to have is around 70 images of 2048x1024 plus others smaller images. 12MB each (decompression is to slow for large image, slower then reading from disk in my test and least color conversion, JPEG are YUV) x 70 = 840MB. And that's for the real minimum I need.
I have a patch that loads about 20 JPG's at 512x512 plus 8 TIFs at 500KB each, on OSX with only 512MB of Ram and this works fine... all using pix_multiimage.
Does pix_multiimage keeping all the images in RAM ? I didn't know that.
Do you know why the CPU goes up when you show an big image coming from a pix_buffer ? It look very strange to me.
I've attached a change to your patch that uses the gemchain render signal as the rotation clock, rather than the metro, Does this also rotate choppily?
Yes it's still choppy, but more regularly choppy instead of random choppy. :-(
Thanks
Etienne
B.
Etienne Desautels wrote:
Hey Ben,
Are these dropped frames your seeing a result of rotating the object via numberbox/PD GUI or from another machine running over -nogui?
No, either. Rotating from metro. (See the patch) I have the same problem on my 667 PowerBook and my Linux pentium 2.6GHz.
I ask because the PD gui is known to have issues and it is easily possible that it is the number box causing this problem and not Gem. Try sending a signal from python over a local socket to pd/Gem running in -nogui.
It will probably work. The problem is the internal timing of PD/metro. It's CPU time base, not real time base. And without threads.
Does it do the same thing?
I'll have to test things on my new linux machine when I get it, but I hope what your saying is not evident. I don't see this happening on OSX. Do you have a test patch?
Here it is !
Disappointed B.
Disappointed E. too.
#N canvas 513 371 819 325 10; #X obj 187 150 gemhead; #X msg 71 9 destroy; #X obj 187 129 tgl 15 0 empty empty empty 0 -6 0 8 -262144 -1 -1 0 1; #X obj 485 157 gemhead; #X obj 414 177 bng 15 250 50 0 empty empty empty 0 -6 0 8 -262144 -1 -1; #X obj 485 264 pix_texture; #X obj 485 287 rectangle 5.333 4; #X msg 414 216 open $1; #X obj 414 195 openpanel; #X obj 485 137 tgl 15 0 empty empty empty 0 -6 0 8 -262144 -1 -1 0 1; #X obj 485 240 pix_image; #X msg 503 186 open /home/etienne/Desktop/froime400.jpg; #X obj 187 198 rotateXYZ; #X obj 289 62 tgl 15 0 empty empty empty 0 -6 0 8 -262144 -1 -1 0 1 ; #X obj 340 62 nbx 5 14 -1e+37 1e+37 0 0 empty empty empty 0 -6 0 10 -262144 -1 -1 0 256; #X obj 289 127 + 1; #X obj 289 105 f 1; #X obj 187 221 rectangle 2 2; #X obj 327 106 nbx 5 14 -1e+37 1e+37 0 0 empty empty empty 0 -6 0 10 -262144 -1 -1 0 256; #X msg 503 210 open /home/etienne/Desktop/thuangS.jpg; #X msg 16 9 create; #X msg 36 30 1; #X msg 98 31 0; #X msg 91 86 frame $1; #X obj 16 113 gemwin 30; #X msg 91 60 30; #X msg 125 60 60; #X obj 289 82 metro 33; #X obj 485 89 loadbang; #X msg 485 113 0; #X text 130 9 1 create the window; #X text 233 29 2 start rotating the square watch it rotating not smoothly! ; #X msg 254 151 0; #X obj 289 151 select 361; #X text 483 48 3 try to load some big textures (1024x1024) with the chain turn on or off watch the square stall; #X connect 0 0 12 0; #X connect 1 0 24 0; #X connect 2 0 0 0; #X connect 3 0 10 0; #X connect 4 0 8 0; #X connect 5 0 6 0; #X connect 7 0 10 0; #X connect 8 0 7 0; #X connect 9 0 3 0; #X connect 10 0 5 0; #X connect 11 0 10 0; #X connect 12 0 17 0; #X connect 13 0 27 0; #X connect 14 0 27 1; #X connect 15 0 33 0; #X connect 16 0 15 0; #X connect 18 0 15 1; #X connect 19 0 10 0; #X connect 20 0 24 0; #X connect 21 0 24 0; #X connect 22 0 24 0; #X connect 23 0 24 0; #X connect 25 0 23 0; #X connect 26 0 23 0; #X connect 27 0 16 0; #X connect 28 0 29 0; #X connect 29 0 9 0; #X connect 32 0 16 0; #X connect 33 0 32 0; #X connect 33 1 16 1; #X connect 33 1 12 3;
:(
Etienne Desautels wrote:
Hello,
On 05-06-05, at 17:14, Tim Blechmann wrote:
hi all ...
> Also, what I would really prefer is to had support in Gem for > LibMPEG2. Do you think it could be difficult to do ? Is it about > creating a filmMPEG2.cpp and filmMPEG2.h files with the good > function calls to the library and add the reference into the > pix_filmNEW.cpp ?
correct. if you like to add libMPEG2-support you are welcome!
stupid question, but what about mplayer support instead of libMPEG2? mplayer is able to play back about every file format ...
just thinking loud ... tim
I know, it would be wonderfull...
But Mplayer is not a library. You can only use it as an application. Mplayer itself use LibMPEG2 to play MPEG2 stream and use other libraries for other formats and codecs (as FFMPEG, XVID, etc).
Unfortunately, I will not add LibMPEG2 to GEM. In fact I'll not use PD/GEM at all for the big project I'm working on. Maybe I'm crazy but the problem is that PD is not multithreaded and work with CPU time and not real time. So if you try in Gem to do something as simple as rotating a square on the screen, it will not rotate regularly. It will stall from time to time. And if you try to load a big texture (1024x1024) the rotation of the square will stall even more badly. And I don't think there's something I can do ? I tried to preload some big textures with pix_buffer but when I display the image it's sucking a lot of CPU. When I say a lot I mean a lot. The more your texture is big the more CPU it's suck. It's very strange. I don't understand why? Anyway, in the mid-term it's not an option for me to preload the images. So now I'm doing the project in Python with pyOpenGL. This way I can schedule redraw in real time, not cpu time, and open my textures in a thread. This way everything play smoothly. But I have a lot more to code ! I wish I could done it with PD/GEM but it's not possible.
One last note: for FFMEPG in Gem, I did try so many things but in the end it never worked. I always have the -2 error meaning that there's an I/O problem?
Anyway thanks for your good work to all the developers of PD and Gem !
Etienne
GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
#N canvas 360 448 907 335 10; #X obj 187 80 gemhead; #X msg 71 9 destroy; #X obj 187 59 tgl 15 0 empty empty empty 0 -6 0 8 -262144 -1 -1 0 1 ; #X obj 485 157 gemhead; #X obj 414 177 bng 15 250 50 0 empty empty empty 0 -6 0 8 -262144 -1 -1; #X obj 485 264 pix_texture; #X obj 485 287 rectangle 5.333 4; #X msg 414 216 open $1; #X obj 414 195 openpanel; #X obj 485 137 tgl 15 0 empty empty empty 0 -6 0 8 -262144 -1 -1 0 1; #X msg 503 186 open /home/etienne/Desktop/froime400.jpg; #X obj 179 258 rotateXYZ; #X obj 274 73 tgl 15 0 empty empty empty 0 -6 0 8 -262144 -1 -1 0 1 ; #X obj 325 73 nbx 5 14 -1e+37 1e+37 0 0 empty empty empty 0 -6 0 10 -262144 -1 -1 0 256; #X obj 237 185 + 1; #X obj 237 163 f 1; #X obj 179 281 rectangle 2 2; #X obj 275 164 nbx 5 14 -1e+37 1e+37 0 0 empty empty empty 0 -6 0 10 -262144 -1 -1 0 256; #X msg 503 210 open /home/etienne/Desktop/thuangS.jpg; #X msg 16 9 create; #X msg 36 30 1; #X msg 98 31 0; #X msg 91 86 frame $1; #X obj 16 113 gemwin 30; #X msg 91 60 30; #X msg 125 60 60; #X obj 274 93 metro 33; #X obj 485 89 loadbang; #X msg 485 113 0; #X text 130 9 1 create the window; #X text 233 29 2 start rotating the square watch it rotating not smoothly! ; #X msg 202 209 0; #X obj 237 209 select 361; #X text 483 48 3 try to load some big textures (1024x1024) with the chain turn on or off watch the square stall; #X obj 485 240 pix_film; #X obj 187 116 t a b; #X text 89 119 Ben magic ->; #X connect 0 0 35 0; #X connect 1 0 23 0; #X connect 2 0 0 0; #X connect 3 0 34 0; #X connect 4 0 8 0; #X connect 5 0 6 0; #X connect 7 0 34 0; #X connect 8 0 7 0; #X connect 9 0 3 0; #X connect 10 0 34 0; #X connect 11 0 16 0; #X connect 12 0 26 0; #X connect 13 0 26 1; #X connect 14 0 32 0; #X connect 15 0 14 0; #X connect 17 0 14 1; #X connect 18 0 34 0; #X connect 19 0 23 0; #X connect 20 0 23 0; #X connect 21 0 23 0; #X connect 22 0 23 0; #X connect 24 0 22 0; #X connect 25 0 22 0; #X connect 27 0 28 0; #X connect 28 0 9 0; #X connect 31 0 15 0; #X connect 32 0 31 0; #X connect 32 1 15 1; #X connect 32 1 11 3; #X connect 34 0 5 0; #X connect 35 0 11 0; #X connect 35 1 15 0;
On Jun 10, 2005, at 8:54 AM, Etienne Desautels wrote:
So I guess you are proposing that the part that opens files be threaded so that loading the file off disk does not stall rendering?
This would have to be pretty carefully done in GEM because a lot of the APIs used are not entirely thread safe. Quicktime is pretty hit or miss with threading as is OpenGL. Just opening the file is not really causing the stall, but rather all of the other setup code and buffer management.
With Quicktime on OSX and Windows there might be a way to get around this which involves using a reference movie. The movie would contain a reference to a set of .mov files and once loaded all of the setup would be done once. This might make switching between clips smoother.
This is impossible for your project because there is not enough RAM to hold all the textures needed?
Yes it's impossible. The minimum acceptable size and number of images I have to have is around 70 images of 2048x1024 plus others smaller images. 12MB each (decompression is to slow for large image, slower then reading from disk in my test and least color conversion, JPEG are YUV) x 70 = 840MB. And that's for the real minimum I need.
How many of those images are loaded at once? If you load every single one using pix_image you will be far beyond VRAM and performance will suffer. Uploading images of that size to the GPU takes some time, which might be the cause of the problem.
cgc
Hi Chris,
On 05-06-10, at 12:33, chris clepper wrote:
On Jun 10, 2005, at 8:54 AM, Etienne Desautels wrote:
So I guess you are proposing that the part that opens files be threaded so that loading the file off disk does not stall rendering?
This would have to be pretty carefully done in GEM because a lot of the APIs used are not entirely thread safe. Quicktime is pretty hit or miss with threading as is OpenGL. Just opening the file is not really causing the stall, but rather all of the other setup code and buffer management.
With Quicktime on OSX and Windows there might be a way to get around this which involves using a reference movie. The movie would contain a reference to a set of .mov files and once loaded all of the setup would be done once. This might make switching between clips smoother.
Yes, that's true.
This is impossible for your project because there is not enough RAM to hold all the textures needed?
Yes it's impossible. The minimum acceptable size and number of images I have to have is around 70 images of 2048x1024 plus others smaller images. 12MB each (decompression is to slow for large image, slower then reading from disk in my test and least color conversion, JPEG are YUV) x 70 = 840MB. And that's for the real minimum I need.
How many of those images are loaded at once?
1 or 2 but with a few smaller ones ( around 120x40). Maybe in the future, the double, because I would like to drive to video output from one computer, one GPU.
If you load every single one using pix_image you will be far beyond VRAM and performance will suffer.
Yes, I know. That's why I need to be able to load it only when I need it but without disrupting the display.
Uploading images of that size to the GPU takes some time, which might be the cause of the problem.
Maybe a part of the problem but I'm not quite sure. If I'm not wrong, AGP 8X have a bandwidth of 2.1GB/s. That's a lot ! I know that's the theoretical speed but at this speed it would take less then 6 ms to load a 2048x1024 (12MB) image from the memory to the GPU. I could not believe that's the main bottleneck. As a proof of this, I did some test in Python with pyOpenGL. At the beginning I had the same problem that I have in PD, the display was stalling when I loaded a 2048x1024 image. Then after, I did some changes and at every change the stall was smaller and smaller. Now I don't have any stall. My first change was reading the image in BMP instead of JPG. More time reading on the disk but no time for decompression and color conversion. My second change was putting the reading of the image in a thread. And my last change was putting the call to load the image just after the redraw of the scene. So, to me, the uploading of the images to the GPU seems to be only a small part of the problem.
Etienne
cgc
Hi Chris,
On 05-06-10, at 12:33, chris clepper wrote:
On Jun 10, 2005, at 8:54 AM, Etienne Desautels wrote:
So I guess you are proposing that the part that opens files be threaded so that loading the file off disk does not stall rendering?
This would have to be pretty carefully done in GEM because a lot of the APIs used are not entirely thread safe. Quicktime is pretty hit or miss with threading as is OpenGL. Just opening the file is not really causing the stall, but rather all of the other setup code and buffer management.
With Quicktime on OSX and Windows there might be a way to get around this which involves using a reference movie. The movie would contain a reference to a set of .mov files and once loaded all of the setup would be done once. This might make switching between clips smoother.
Yes, that's true.
This is impossible for your project because there is not enough RAM to hold all the textures needed?
Yes it's impossible. The minimum acceptable size and number of images I have to have is around 70 images of 2048x1024 plus others smaller images. 12MB each (decompression is to slow for large image, slower then reading from disk in my test and least color conversion, JPEG are YUV) x 70 = 840MB. And that's for the real minimum I need.
How many of those images are loaded at once?
1 or 2 but with a few smaller ones ( around 120x40). Maybe in the future, the double, because I would like to drive to video output from one computer, one GPU.
If you load every single one using pix_image you will be far beyond VRAM and performance will suffer.
Yes, I know. That's why I need to be able to load it only when I need it but without disrupting the display.
Uploading images of that size to the GPU takes some time, which might be the cause of the problem.
Maybe a part of the problem but I'm not quite sure. If I'm not wrong, AGP 8X have a bandwidth of 2.1GB/s. That's a lot ! I know that's the theoretical speed but at this speed it would take less then 6 ms to load a 2048x1024 (12MB) image from the memory to the GPU. I could not believe that's the main bottleneck. As a proof of this, I did some test in Python with pyOpenGL. At the beginning I had the same problem that I have in PD, the display was stalling when I loaded a 2048x1024 image. Then after, I did some changes and at every change the stall was smaller and smaller. Now I don't have any stall. My first change was reading the image in BMP instead of JPG. More time reading on the disk but no time for decompression and color conversion. My second change was putting the reading of the image in a thread. And my last change was putting the call to load the image just after the redraw of the scene. So, to me, the uploading of the images to the GPU seems to be only a small part of the problem.
Etienne
cgc
Tim Blechmann wrote:
hi all ...
Also, what I would really prefer is to had support in Gem for LibMPEG2. Do you think it could be difficult to do ? Is it about creating a filmMPEG2.cpp and filmMPEG2.h files with the good function calls to the library and add the reference into the pix_filmNEW.cpp ?
correct. if you like to add libMPEG2-support you are welcome!
stupid question, but what about mplayer support instead of libMPEG2? mplayer is able to play back about every file format ...
hmm, you mean like mjpegtools ? letting mplayer write into a fifo and read that back ?
the only problem i see is, how one would control the mplayer.... (but i haven't looked closer)
mfg.asdr.a IOhannes
just thinking loud ... tim
On 05-06-09, at 13:59, IOhannes m zmoelnig wrote:
Tim Blechmann wrote:
hi all ...
Also, what I would really prefer is to had support in Gem for LibMPEG2. Do you think it could be difficult to do ? Is it about creating a filmMPEG2.cpp and filmMPEG2.h files with the good function calls to the library and add the reference into the pix_filmNEW.cpp ?
correct. if you like to add libMPEG2-support you are welcome!
stupid question, but what about mplayer support instead of libMPEG2? mplayer is able to play back about every file format ...
hmm, you mean like mjpegtools ? letting mplayer write into a fifo and read that back ?
the only problem i see is, how one would control the mplayer.... (but i haven't looked closer)
Sébastien Roy from the TOT project (PixelTango, etc.) at SAT is using this approach for his project (LightTwist). He's using a bit of PD but it use mostly OpenSceneGraph for the 3D part and is own C code. But he's using a shared buffer instead of a FIFO. A FIFO could be to slow because all the decompress data have to be write and read from the disk. With the shared buffer the decompress data is written and read in the memory and he got really good results. But I don't know how he control Mplayer ? I will ask.
Etienne
mfg.asdr.a IOhannes
just thinking loud ... tim
stupid question, but what about mplayer support instead of libMPEG2? mplayer is able to play back about every file format ...
hmm, you mean like mjpegtools ? letting mplayer write into a fifo and read that back ?
the only problem i see is, how one would control the mplayer.... (but i haven't looked closer)
well, mplayer does have some kind of command line interface ... (see slave.txt) ...
on the other hand, trying to write a jack-transport aware python script controlling mplayer i figured out that the synchronisation sucks ...
still, in the contrary of Gem, mplayer can play back dvd vob files, without recompiling with other libraries or crashing ...
cheers ... tim
Tim Blechmann wrote:
still, in the contrary of Gem, mplayer can play back dvd vob files, without recompiling with other libraries or crashing ...
since i don't have problems playing back VOB-files with a "normally" compiled Gem (not the Debian package though), i would like to hear other users experiences about that:
i am using: 800MHz-P3, 256MB ram, kernel-2.6.10, pd-0.38-4, gem-CVS(HEAD)
could be your DVD could be from the wrong region ??
mfg.asd.r IOhannes
could be your DVD could be from the wrong region ??
i doubt that ...
cheers ... tim
Hey all,
This has been sitting in my inbox for way too long!
So again I'm in a situation where I want to load mpeg2 files in Gem. Of course ffmpeg handles this fine, and plays well enough for what I want. ffplay plays the file fine and Gem loads the file "with -1 frames" but does not interpret the file properly and we get all kinds of distortion and artifacts, looks like nothing is getting decoded properly.
I tried changing the extension and playing with the new open [blah] loader stuff without any luck, all combinations do not load the file that at all resebles the output of ffplay.
Looking at the message below, where should I add these linker flags?
Although pidip has dropped ffmpeg, still seems to me that it works very well and supports many codecs. So how do we make the ffmpeg loader in Gem as functional as ffplay?
I'll take a look at the ffplay source, but considering my experience not much is going to happen with the result.
B>
IOhannes m zmoelnig wrote:
Etienne Desautels wrote:
Hi list,
So here's my questions. I tried to compile Gem with FFMPEG support and it always fail in one manner or another. Or it doesn't build at all, or it build but when I load Gem I got this error message (undefined symbol: ogg_stream_init Gem: can't load library), or everything seems OK but
hmm, are you using the CVS version of Gem ? there are some better linking options in the CVS-version (basically it adds the output of "ffmpeg-config --plugin-libs" which adds more libraries than just libavcodec and libavformat) if not, you will probably have to add these flags by hand: "-lavformat_pic -lavcodec_pic -lvorbis -lvorbisenc -ldts_pic -la52"
Also, what I would really prefer is to had support in Gem for LibMPEG2. Do you think it could be difficult to do ? Is it about creating a filmMPEG2.cpp and filmMPEG2.h files with the good function calls to the library and add the reference into the pix_filmNEW.cpp ?
correct. if you like to add libMPEG2-support you are welcome!
One last thing, I was not able to use FSAA. My video card is an ATI 9600 and I'm using the binary driver with x.org. I put the lines to enable FSAA in my X86-config-4 file without any success. Any ideas ?
sorry, no experience with Xorg
mfg.as.dr IOhannes
GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
-
B. Bogart -
chris clepper -
Etienne Desautels -
g�nter geiger -
IOhannes m zmoelnig -
Tim Blechmann