Hello all, sorry for the cross-posting,
Back to the old codec issue here. I was hapiliy working with photo-jpg compressed quicktime files for the last couple years. Now I'm working on an installation that I'm aiming to make using only open-source software.
My "orginal" footage is a sequence of 300,000 jpeg frames (80% quality) And all I need is a video file that matches the jpg quality of the frames, and most importantly is seekable and loadable in Gem.
So what I want to do is create a quicktime file, using photo-jpeg as the codec. Simple eh? hmmm.
My first try was:
ffmpeg -i %d.jpg -sameq -vcodec ljpeg output.mov
Which created the file fine (where, FYI, quicktime pro died miserably) but was very big (which is ok) but I could not get vlc,mplayer,totem to load it. ffmpeg printed something about "MS style codec tag" when creating the file. so I also tried:
ffmpeg -i %d.jpg -sameq -vcodec mjpeg output.mov
which did about the same thing, so I gave up on quicktime and tried a couple AVIs:
ffmpeg -i %d.jpg -sameq -vcodec mjpeg output.avi ffmpeg -i %d.jpg -sameq -vcodec ljpeg output.avi
A little better, many of my Gem decoders loaded the file, but not a single one was able to make it seekable, I tried all the decoders... all return -1 frames.
So right now I'm building a mpeg2 to try out, though I know decoding is going to be less efficient.
So my final questions are:
1. What codec should I use for 300,000 frames (14GB or so) and is well suited to build from jpg frames? What container?
2. How can I create the [answer to #1] using ffmpeg or some other linux land app?
Thank you!
Oh and Johannes, I do have the v4l_compat module loaded, so I don't know why xawtv reads my v4l2 video stream but Gem does not. Any ideas?
<B>
B. Bogart wrote:
Hello all, sorry for the cross-posting,
So my final questions are:
- What codec should I use for 300,000 frames (14GB or so) and is well
suited to build from jpg frames? What container?
as marc has pointed out: mjpegtools
you can also use ppmtompeg (on debian this is part of netpbm) to create an mpeg out of a series of images (despite of the "ppm" in the name they can also be jpegs)
Oh and Johannes, I do have the v4l_compat module loaded, so I don't know why xawtv reads my v4l2 video stream but Gem does not. Any ideas?
so the current problem is freezing (not just being unable to connect to the driver) ? are you using pd's realtime flag ?
you could try running everything in gdb and then kill the process when it freezes and then get a backtrace to see where it was when it froze.
mfg.juz.ssd IOhannes
I am not using -rt at the moment (may later depending on how the patch eats up CPU)
And yes the problem is the freeze, not attaching to the driver, I never do see a frame of video though, from black to frozen. The specific cause of the freeze is sending the "mode ntsc 1" message. the pix_video in a patch does NOT freeze pd when loading the patch, but it does not show any video either. Once I try and change the video port that is when the freeze happens. I can change the video mode before creating the gemwin, no freeze, but then when I try and change it while rendering, it freezes. Freezes on "mode ntsc $1" where $1 is 0-2. Seems I never see any video get textured. Console seems to be happy printing stuff like:
pix_videoNEW: starting transfer GEM: pix_video: Opened video connection 12
When pix_video loads and there is both v4l and ieee1394 on the machine, which is "driver 0"? v4l is reported in the console first.
Here is the bt after killing pd
Program received signal SIGTERM, Terminated. [Switching to Thread 1076267520 (LWP 21548)] 0xffffe410 in __kernel_vsyscall () (gdb) bt #0 0xffffe410 in __kernel_vsyscall () #1 0x40128e48 in pthread_join () from /lib/tls/i686/cmov/libpthread.so.0 #2 0x44bec81d in videoV4L::stopTransfer () from /usr/lib/pd/extra/Gem.pd_linux #3 0x44bec9d6 in videoV4L::setNorm () from /usr/lib/pd/extra/Gem.pd_linux #4 0x44be81c5 in pix_videoNEW::normMess () from /usr/lib/pd/extra/Gem.pd_linux #5 0x44be898a in pix_videoNEW::modeMessCallback () from /usr/lib/pd/extra/Gem.pd_linux #6 0x0809ba47 in pd_typedmess () #7 0x00000000 in ?? () #8 0x080ecce6 in _IO_stdin_used () #9 0x00000022 in ?? () #10 0xffffffff in ?? () #11 0x00000000 in ?? () #12 0x00000000 in ?? () #13 0x00000000 in ?? () #14 0x00000000 in ?? () #15 0x080eccff in _IO_stdin_used () #16 0x08383bf0 in ?? () #17 0x40193251 in funlockfile () from /lib/tls/i686/cmov/libc.so.6 Previous frame inner to this frame (corrupt stack?)
Any tests/suggestions would be amazing.
I played around with it more and got a similar trace but w/ an actual crash caused by sending "0, destroy" to gemwin while I'm seeing just the black texture (and I tried it again and this time it did not segfault, but gave the same trace as above):
Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 1076267520 (LWP 22042)] ---Type <return> to continue, or q <return> to quit--- 0x40127af8 in __free_tcb () from /lib/tls/i686/cmov/libpthread.so.0 (gdb) bt #0 0x40127af8 in __free_tcb () from /lib/tls/i686/cmov/libpthread.so.0 #1 0x40128e8d in pthread_join () from /lib/tls/i686/cmov/libpthread.so.0 #2 0x44bec81d in videoV4L::stopTransfer () from /usr/lib/pd/extra/Gem.pd_linux #3 0x44be80ac in pix_videoNEW::stopRendering () from /usr/lib/pd/extra/Gem.pd_linux #4 0x44b1417e in GemBase::gem_startstopMess () from /usr/lib/pd/extra/Gem.pd_linux #5 0x44b143e2 in GemBase::gem_MessCallback () from /usr/lib/pd/extra/Gem.pd_linux #6 0x0809ba47 in pd_typedmess ()
Oh! I got a single frame of what looks like 96x96 pixel cable noise, but only one frame. First time I saw anything at all getting textured. Doing the same thing a second time I'm not getting anywhere.
Thats enough for now, back to serial programming the camera.
I'll do any tests needed to get this working, just tell me what to do.
Let me know if it would help to meet on IRC.
Thanks Johannes.
b.
IOhannes m zmoelnig wrote:
Oh and Johannes, I do have the v4l_compat module loaded, so I don't know why xawtv reads my v4l2 video stream but Gem does not. Any ideas?
so the current problem is freezing (not just being unable to connect to the driver) ? are you using pd's realtime flag ?
you could try running everything in gdb and then kill the process when it freezes and then get a backtrace to see where it was when it froze.
mfg.juz.ssd IOhannes
Hey all,
I thought what the heck, lets see if I can see my tuner cards in pdp_v4l. So I get the same freeze as I got in Gem:
(gdb) bt #0 0xffffe410 in __kernel_vsyscall () #1 0x40128e48 in pthread_join () from /lib/tls/i686/cmov/libpthread.so.0 #2 0x42a86440 in pdp_sdl_setup () from /usr/lib/pd/extra/pdp.pd_linux
so my pthreads are killing me...
Help, What do I do?
debian sarge w/ demudi and unstable packages. (pthread is stable/sarge) 2.6.8 32bit kernel on an AMD64.
Thanks in advance, I need to fix this for an installation in 1 month. (and a lot of other complex stuff to do as well)
B.
B. Bogart wrote:
I am not using -rt at the moment (may later depending on how the patch eats up CPU)
And yes the problem is the freeze, not attaching to the driver, I never do see a frame of video though, from black to frozen. The specific cause of the freeze is sending the "mode ntsc 1" message. the pix_video in a patch does NOT freeze pd when loading the patch, but it does not show any video either. Once I try and change the video port that is when the freeze happens. I can change the video mode before creating the gemwin, no freeze, but then when I try and change it while rendering, it freezes. Freezes on "mode ntsc $1" where $1 is 0-2. Seems I never see any video get textured. Console seems to be happy printing stuff like:
pix_videoNEW: starting transfer GEM: pix_video: Opened video connection 12
When pix_video loads and there is both v4l and ieee1394 on the machine, which is "driver 0"? v4l is reported in the console first.
Here is the bt after killing pd
Program received signal SIGTERM, Terminated. [Switching to Thread 1076267520 (LWP 21548)] 0xffffe410 in __kernel_vsyscall () (gdb) bt #0 0xffffe410 in __kernel_vsyscall () #1 0x40128e48 in pthread_join () from /lib/tls/i686/cmov/libpthread.so.0 #2 0x44bec81d in videoV4L::stopTransfer () from /usr/lib/pd/extra/Gem.pd_linux #3 0x44bec9d6 in videoV4L::setNorm () from /usr/lib/pd/extra/Gem.pd_linux #4 0x44be81c5 in pix_videoNEW::normMess () from /usr/lib/pd/extra/Gem.pd_linux #5 0x44be898a in pix_videoNEW::modeMessCallback () from /usr/lib/pd/extra/Gem.pd_linux #6 0x0809ba47 in pd_typedmess () #7 0x00000000 in ?? () #8 0x080ecce6 in _IO_stdin_used () #9 0x00000022 in ?? () #10 0xffffffff in ?? () #11 0x00000000 in ?? () #12 0x00000000 in ?? () #13 0x00000000 in ?? () #14 0x00000000 in ?? () #15 0x080eccff in _IO_stdin_used () #16 0x08383bf0 in ?? () #17 0x40193251 in funlockfile () from /lib/tls/i686/cmov/libc.so.6 Previous frame inner to this frame (corrupt stack?)
Any tests/suggestions would be amazing.
I played around with it more and got a similar trace but w/ an actual crash caused by sending "0, destroy" to gemwin while I'm seeing just the black texture (and I tried it again and this time it did not segfault, but gave the same trace as above):
Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 1076267520 (LWP 22042)] ---Type <return> to continue, or q <return> to quit--- 0x40127af8 in __free_tcb () from /lib/tls/i686/cmov/libpthread.so.0 (gdb) bt #0 0x40127af8 in __free_tcb () from /lib/tls/i686/cmov/libpthread.so.0 #1 0x40128e8d in pthread_join () from /lib/tls/i686/cmov/libpthread.so.0 #2 0x44bec81d in videoV4L::stopTransfer () from /usr/lib/pd/extra/Gem.pd_linux #3 0x44be80ac in pix_videoNEW::stopRendering () from /usr/lib/pd/extra/Gem.pd_linux #4 0x44b1417e in GemBase::gem_startstopMess () from /usr/lib/pd/extra/Gem.pd_linux #5 0x44b143e2 in GemBase::gem_MessCallback () from /usr/lib/pd/extra/Gem.pd_linux #6 0x0809ba47 in pd_typedmess ()
Oh! I got a single frame of what looks like 96x96 pixel cable noise, but only one frame. First time I saw anything at all getting textured. Doing the same thing a second time I'm not getting anywhere.
Thats enough for now, back to serial programming the camera.
I'll do any tests needed to get this working, just tell me what to do.
Let me know if it would help to meet on IRC.
Thanks Johannes.
b.
IOhannes m zmoelnig wrote:
Oh and Johannes, I do have the v4l_compat module loaded, so I don't know why xawtv reads my v4l2 video stream but Gem does not. Any ideas?
so the current problem is freezing (not just being unable to connect to the driver) ? are you using pd's realtime flag ?
you could try running everything in gdb and then kill the process when it freezes and then get a backtrace to see where it was when it froze.
mfg.juz.ssd IOhannes
GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
Thanks for the suggestions.
I'm not sure about using a mpeg for the following reasons:
#1. was does the interframe compression thing work, in other words how does it compare when seeking to a quicktime with photo-jpg codec?
#2. I'm damn confused about the seeking (see below)
So I have mpeg1 and mpeg3 on my machine,
% "open alea.mpg 4" gives me a seekable file. GEM: pix_film: Loaded file: /home/bbogart/pd-stuff/Gem/examples/data/alea.mpg with 31 frames (320x240) % file alea.mpg alea.mpg: MPEG sequence, v1, progressive Y'CbCr 4:2:0 video, 320x240, 4:3, 30 fps
% "open test.mpg 4" gives me a unseekable file. GEM: pix_film: Loaded file: /raid0/membrane/test.mpg with 0 frames (640x480) % file test.mpg test.mpg: MPEG sequence, v1, system multiplex
test.mpg was created with ffmpeg.
So if an mpeg performs as well as a quicktime with photo-jpeg than can someone give me an example of how to create an mov that will be seekable in Gem using mjpegtools or transcode? Since obviously its not difficult to make an unseekable one...
Aside: What id the different between mjpeg and photojpeg as a video codec? ffmpeg reports my sequence of jpegs as "mjpeg"...
Ok, I'm installing mjpegtools and netpbm.
B.
IOhannes m zmoelnig wrote:
B. Bogart wrote:
Hello all, sorry for the cross-posting,
So my final questions are:
- What codec should I use for 300,000 frames (14GB or so) and is well
suited to build from jpg frames? What container?
as marc has pointed out: mjpegtools
you can also use ppmtompeg (on debian this is part of netpbm) to create an mpeg out of a series of images (despite of the "ppm" in the name they can also be jpegs)
On Aug 30, 2005, at 10:05 PM, B. Bogart wrote:
Thanks for the suggestions.
I'm not sure about using a mpeg for the following reasons:
#1. was does the interframe compression thing work, in other words how does it compare when seeking to a quicktime with photo-jpg codec?
You want to avoid interframe compression at all costs. In order to grab a random frame in an interframe codec the nearest previous keyframe must be found and all interframes between it and the requested frame are decoded. You could get lucky and hit a keyframe or be very unlucky and hit the frame right before one.
Interframe compression techniques are used almost exclusively for final content distribution (web, DVD, ATSC). Acquisition and production codec never use this with the exception of HDV (which is MPEG2).
Aside: What id the different between mjpeg and photojpeg as a video codec?
mjpeg is motion-jpeg which is for interlaced two field video. Each field (50 or 60 per second) is a compressed with jpeg techniques.
ffmpeg reports my sequence of jpegs as "mjpeg"...
It is mistaken if your sequence is single progressive frames, which it almost certainly is.
cgc
Hi all,
Thanks for the wealth of info.
I made a couple mpegs and TOCs and it works fine. I decided to create GOPs of only a single frame to force the lack of interframe compression.
So it *almost* works in Gem, almost eh...
Well I can seek only forwards, not backwards.
Lets call the current frame Fc and the previous frame Fp:
if Fp < Fc seeking works fine if Fp > Fc the mpeg immediatly jumps to the first frame and sticks on the first frame for as long as this condition is true.
Maybe I need a 2 frame GOP to get bidirectionality?
I've been unable to seek any mpeg at all without using the mpeg3toc thing, Including my single frame GOP, they just scrub forward as you twiddle the frame number.
We'll I'm close anyhow. off to look at lvs-studio...
Thanks again,
B.
chris clepper wrote:
On Aug 30, 2005, at 10:05 PM, B. Bogart wrote:
Thanks for the suggestions.
I'm not sure about using a mpeg for the following reasons:
#1. was does the interframe compression thing work, in other words how does it compare when seeking to a quicktime with photo-jpg codec?
You want to avoid interframe compression at all costs. In order to grab a random frame in an interframe codec the nearest previous keyframe must be found and all interframes between it and the requested frame are decoded. You could get lucky and hit a keyframe or be very unlucky and hit the frame right before one.
Interframe compression techniques are used almost exclusively for final content distribution (web, DVD, ATSC). Acquisition and production codec never use this with the exception of HDV (which is MPEG2).
Aside: What id the different between mjpeg and photojpeg as a video codec?
mjpeg is motion-jpeg which is for interlaced two field video. Each field (50 or 60 per second) is a compressed with jpeg techniques.
ffmpeg reports my sequence of jpegs as "mjpeg"...
It is mistaken if your sequence is single progressive frames, which it almost certainly is.
cgc
PD-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Correction:
I just created a TOC from a mpeg2enc -f 1 file and I CAN seek backwards!
I was confused because a -f 0 mpeg (mpeg1) did not allow this, and "file" reports my -f 1 (mpeg2) stream as a v1 stream... blech.
Anyhow I need to tweak the quality a bit and we're in business.
For the archives this is the command I used to create the mpeg2 from frames using mjpegtools:
jpeg2yuv -f 25 -I p -v 0 -j ./test/link/%d.jpg | mpeg2enc -o test3.mpg -g 1 -G 1 -f 1 && mpeg3toc test3.mpg test3.toc
which looks pretty ugly for the record, even being 600MB (small)
Thanks all.
B>
B. Bogart wrote:
Hi all,
Thanks for the wealth of info.
I made a couple mpegs and TOCs and it works fine. I decided to create GOPs of only a single frame to force the lack of interframe compression.
So it *almost* works in Gem, almost eh...
Well I can seek only forwards, not backwards.
Lets call the current frame Fc and the previous frame Fp:
if Fp < Fc seeking works fine if Fp > Fc the mpeg immediatly jumps to the first frame and sticks on the first frame for as long as this condition is true.
Maybe I need a 2 frame GOP to get bidirectionality?
I've been unable to seek any mpeg at all without using the mpeg3toc thing, Including my single frame GOP, they just scrub forward as you twiddle the frame number.
We'll I'm close anyhow. off to look at lvs-studio...
Thanks again,
B.
chris clepper wrote:
On Aug 30, 2005, at 10:05 PM, B. Bogart wrote:
Thanks for the suggestions.
I'm not sure about using a mpeg for the following reasons:
#1. was does the interframe compression thing work, in other words how does it compare when seeking to a quicktime with photo-jpg codec?
You want to avoid interframe compression at all costs. In order to grab a random frame in an interframe codec the nearest previous keyframe must be found and all interframes between it and the requested frame are decoded. You could get lucky and hit a keyframe or be very unlucky and hit the frame right before one.
Interframe compression techniques are used almost exclusively for final content distribution (web, DVD, ATSC). Acquisition and production codec never use this with the exception of HDV (which is MPEG2).
Aside: What id the different between mjpeg and photojpeg as a video codec?
mjpeg is motion-jpeg which is for interlaced two field video. Each field (50 or 60 per second) is a compressed with jpeg techniques.
ffmpeg reports my sequence of jpegs as "mjpeg"...
It is mistaken if your sequence is single progressive frames, which it almost certainly is.
cgc
PD-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
PD-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Correction #2:
mpeg2enc -f 0 is mpeg1 mpeg2enc -f 1 is mpeg1 to VCD spec (huh?)
I'm trying -f 3 (mpeg2) now...
b.
B. Bogart wrote:
Correction:
I just created a TOC from a mpeg2enc -f 1 file and I CAN seek backwards!
I was confused because a -f 0 mpeg (mpeg1) did not allow this, and "file" reports my -f 1 (mpeg2) stream as a v1 stream... blech.
Anyhow I need to tweak the quality a bit and we're in business.
For the archives this is the command I used to create the mpeg2 from frames using mjpegtools:
jpeg2yuv -f 25 -I p -v 0 -j ./test/link/%d.jpg | mpeg2enc -o test3.mpg -g 1 -G 1 -f 1 && mpeg3toc test3.mpg test3.toc
which looks pretty ugly for the record, even being 600MB (small)
Thanks all.
B>
B. Bogart wrote:
Hi all,
Thanks for the wealth of info.
I made a couple mpegs and TOCs and it works fine. I decided to create GOPs of only a single frame to force the lack of interframe compression.
So it *almost* works in Gem, almost eh...
Well I can seek only forwards, not backwards.
Lets call the current frame Fc and the previous frame Fp:
if Fp < Fc seeking works fine if Fp > Fc the mpeg immediatly jumps to the first frame and sticks on the first frame for as long as this condition is true.
Maybe I need a 2 frame GOP to get bidirectionality?
I've been unable to seek any mpeg at all without using the mpeg3toc thing, Including my single frame GOP, they just scrub forward as you twiddle the frame number.
We'll I'm close anyhow. off to look at lvs-studio...
Thanks again,
B.
chris clepper wrote:
On Aug 30, 2005, at 10:05 PM, B. Bogart wrote:
Thanks for the suggestions.
I'm not sure about using a mpeg for the following reasons:
#1. was does the interframe compression thing work, in other words how does it compare when seeking to a quicktime with photo-jpg codec?
You want to avoid interframe compression at all costs. In order to grab a random frame in an interframe codec the nearest previous keyframe must be found and all interframes between it and the requested frame are decoded. You could get lucky and hit a keyframe or be very unlucky and hit the frame right before one.
Interframe compression techniques are used almost exclusively for final content distribution (web, DVD, ATSC). Acquisition and production codec never use this with the exception of HDV (which is MPEG2).
Aside: What id the different between mjpeg and photojpeg as a video codec?
mjpeg is motion-jpeg which is for interlaced two field video. Each field (50 or 60 per second) is a compressed with jpeg techniques.
ffmpeg reports my sequence of jpegs as "mjpeg"...
It is mistaken if your sequence is single progressive frames, which it almost certainly is.
cgc
PD-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
PD-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
PD-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
B. Bogart wrote:
Thanks for the suggestions.
I'm not sure about using a mpeg for the following reasons:
#1. was does the interframe compression thing work, in other words how does it compare when seeking to a quicktime with photo-jpg codec?
#2. I'm damn confused about the seeking (see below)
as chris has pointed out, interframe compression can be a crux. however: solution a: make each frame a keyframe (should be possible with the encoder of your choice) solution b: don't care about keyframes at all, use an external tool to create a TOC of your mpeg, so that the decoder knows where to look for keyframes (mpeg3toc of the mpeg3-utils package) and load the toc-file into Gem and seek the file.
of course, solution b is more cpu and disk intensive, so you might still want to avoid it. it relies on libmpeg3 (imo libmpeg1 is unusable anyhow)
mfg.ads.r IOhannes
-
B. Bogart -
chris clepper -
IOhannes m zmoelnig