Hello,
I have been chosen for Google Summer of Code and am working on a video input-output framework for GEM. This framework should make it possible to load different video encoders/decoders at runtime so that there is no need for them to be available at compiling time. So if you have some suggestions for my project it would be nice to let me know.
Thomas
Why are you doing this? I can maybe see the utility on Linux but what you propose makes no sense on OSX or Windows.
On 6/12/07, Thomas Holzmann holzi1@gmx.at wrote:
Hello,
I have been chosen for Google Summer of Code and am working on a video input-output framework for GEM. This framework should make it possible to load different video encoders/decoders at runtime so that there is no need for them to be available at compiling time. So if you have some suggestions for my project it would be nice to let me know.
Thomas
GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
Why do you think it makes no sense for OSX or Windows? The problem is the same on every platform, isn it?
chris clepper schrieb:
Why are you doing this? I can maybe see the utility on Linux but what you propose makes no sense on OSX or Windows.
On 6/12/07, Thomas Holzmann holzi1@gmx.at wrote:
Hello,
I have been chosen for Google Summer of Code and am working on a video input-output framework for GEM. This framework should make it possible to load different video encoders/decoders at runtime so that there is no need for them to be available at compiling time. So if you have some suggestions for my project it would be nice to let me know.
Thomas
GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
chris clepper wrote:
Why are you doing this?
he is basically doing this, because i told him so. (you could have guessed;-)) we (that is: the iem) proposed this project as a gsoc-project, and it was accepted.
I can maybe see the utility on Linux but what you propose makes no sense on OSX or Windows.
it makes sense on windows, as there are already 2 (or 3, if you count the old vfw/avi stuff) APIs (directshow, quicktime).
on os-x things are a bit different, as there is basically just one API.
nevertheless, the platform that will benefit most from it will be linux.
furthermore, the proposed videoIO framework might (in a 2nd step) be extended to be usable not only by Gem, but by other libraries (pdp, GridFlow, probably pure-pd once it gets that far...) too.
the main reason for doing so is to separate the grabbing/outputting code from the Gem core, in order to reduce dependencies on dynamic libraries. i don't know how things are now (because i spend so little time in front of w32-machines), but iirc adding QuickTime support to the w32 (which was a great thing to do), also added the need to have the quicktime-framework installed, in order to use Gem at all (but i might be completely mistaken here, so forgive my ignorance).
On 6/12/07, Thomas Holzmann holzi1@gmx.at wrote:
Hello,
I have been chosen for Google Summer of Code and am working on a video input-output framework for GEM. This framework should make it possible to load different video encoders/decoders at runtime so that there is no need for them to be available at compiling time.
the framework should cover: - READING: random access reading of media (e.g. movie files; images, frameserver read access) - READING: stream readers (camera input, web video-stream input) - WRITING: stream writers (movie files, video out (e.g. firewire), web video streaming, images, sending data to frameservers)
so basically it should provide the grand unified API for r/w access of image data.
this should (in theory) make it easier to add r/w capabilities of image-data to all sorts of objects, not just the [pix_video], [pix_film], [pix_write], [pix_record]: for instance it might make the [pix_share_*] objects obsolete (could be covered by [pix_streamIN]/[pix_streamOUT]), or [pix_buffer] could very easily have read and write capabilities,....
So if you have some suggestions for my project it would be nice to let me know.
while we have some rather ok knowledge on how things work on linux, it would be good if people familiar with os-x and windows (chris, that's you!) shared some of there knowledge so that the framework can be designed in a way that it will be possible to add QuickTime and DirectShow support efficiently once someone is in the mood for doing so.
since i think that these APIs are fundamentally different from most (if not all) linux APIs, it would be good to not repeat the filmNEW and videoNEW problems, that kind of mapped not so easy to QT/DS as i would have wished.
so chris, even if you are not interested in this project and have better things to do, it would be very nice if you could warn thomas of possible pitfalls with his project.
in the end we might all benefit from such a thing (or not; but we will only know afterwards)
mfga.sdr IOhannes
My general thoughts on this are two points:
1) Of all the things to work on in GEM I would place a complete overhaul of the pix_film/video near the bottom of the list. My number one request would be for documentation especially of the more advanced features. Also, it would be great to have someone look at the things Jamie was working on and continue those. GLEW support would be nice to have but that also might require a lot of code changes. Finally, having someone who really knows Windows and is willing to work on that platform is much needed.
2) This directly affects the code I use most to make a living. Since the work here is designed for very long term installation the testing process is extensive and lengthy. Depending on how large the changes are the testing could require months of labor and dedicated testing machines. This would cost a fair amount of money and the end result would be something that works as well as it does today rather than a great improvement.
If the goal is to really do this 'right' in a final way, there has to be someone far more proficient in Windows development than either of us. I don't hate Windows but I just do not have enough background in development on the platform. Case in point: the DirectShow movie code only works because a programmer was hired to come in and help me with it. I didn't have the time to learn all about COM (whatever the hell that is) and how that programming model relates to DirectShow. The API itself is a confusing set of black boxes that requires too much hoping and praying that the invisible wires behind the scenes actually work like they should. I know enough about DirectShow to get things working but would not be that comfortable making large changes.
I would like to see a fairly comprehensive and detailed proposal of how this is going to work and then I can give more specifics.
On 6/13/07, IOhannes m zmoelnig zmoelnig@iem.at wrote:
chris clepper wrote:
Why are you doing this?
he is basically doing this, because i told him so. (you could have guessed;-)) we (that is: the iem) proposed this project as a gsoc-project, and it was accepted.
I can maybe see the utility on Linux but what you propose makes no sense on OSX or Windows.
it makes sense on windows, as there are already 2 (or 3, if you count the old vfw/avi stuff) APIs (directshow, quicktime).
on os-x things are a bit different, as there is basically just one API.
nevertheless, the platform that will benefit most from it will be linux.
furthermore, the proposed videoIO framework might (in a 2nd step) be extended to be usable not only by Gem, but by other libraries (pdp, GridFlow, probably pure-pd once it gets that far...) too.
the main reason for doing so is to separate the grabbing/outputting code from the Gem core, in order to reduce dependencies on dynamic libraries. i don't know how things are now (because i spend so little time in front of w32-machines), but iirc adding QuickTime support to the w32 (which was a great thing to do), also added the need to have the quicktime-framework installed, in order to use Gem at all (but i might be completely mistaken here, so forgive my ignorance).
On 6/12/07, Thomas Holzmann holzi1@gmx.at wrote:
Hello,
I have been chosen for Google Summer of Code and am working on a video input-output framework for GEM. This framework should make it possible to load different video encoders/decoders at runtime so that there is no need for them to be available at compiling time.
the framework should cover:
- READING: random access reading of media (e.g. movie files; images,
frameserver read access)
- READING: stream readers (camera input, web video-stream input)
- WRITING: stream writers (movie files, video out (e.g. firewire), web
video streaming, images, sending data to frameservers)
so basically it should provide the grand unified API for r/w access of image data.
this should (in theory) make it easier to add r/w capabilities of image-data to all sorts of objects, not just the [pix_video], [pix_film], [pix_write], [pix_record]: for instance it might make the [pix_share_*] objects obsolete (could be covered by [pix_streamIN]/[pix_streamOUT]), or [pix_buffer] could very easily have read and write capabilities,....
So if you have some suggestions for my project it would be nice to let me know.
while we have some rather ok knowledge on how things work on linux, it would be good if people familiar with os-x and windows (chris, that's you!) shared some of there knowledge so that the framework can be designed in a way that it will be possible to add QuickTime and DirectShow support efficiently once someone is in the mood for doing so.
since i think that these APIs are fundamentally different from most (if not all) linux APIs, it would be good to not repeat the filmNEW and videoNEW problems, that kind of mapped not so easy to QT/DS as i would have wished.
so chris, even if you are not interested in this project and have better things to do, it would be very nice if you could warn thomas of possible pitfalls with his project.
in the end we might all benefit from such a thing (or not; but we will only know afterwards)
mfga.sdr IOhannes
chris clepper wrote:
My general thoughts on this are two points:
- Of all the things to work on in GEM I would place a complete
overhaul of the pix_film/video near the bottom of the list. My number one request would be for documentation especially of the more advanced features. Also, it would be great to have someone look at the things Jamie was working on and continue those. GLEW support would be nice to have but that also might require a lot of code changes. Finally,
i agree with most points here. the problem is, that several projects related to Gem have been submitted (featuring glew, the videoIO framework, threading,...) and this framework is the one that was chosen (to be honest: it was the only gem-related project that someone was willing to do).
having someone who really knows Windows and is willing to work on that platform is much needed.
and this is certainly true and sad.
- This directly affects the code I use most to make a living. Since
the work here is designed for very long term installation the testing process is extensive and lengthy. Depending on how large the changes are the testing could require months of labor and dedicated testing machines. This would cost a fair amount of money and the end result would be something that works as well as it does today rather than a great improvement.
i see your point. otoh, if GridFlow partakes (and probably pdp) a lot more people will actually be doing the testing.
obviously it is not the plan to render previous work useless. probably we can find a way to do a transition smoothly (without any NEW and OLD adhoc solutions)
I would like to see a fairly comprehensive and detailed proposal of how this is going to work and then I can give more specifics.
ok that is great.
mfga.sdr IOhannes
(french and english)
Bonjour à tous !
voici le pix_polock ! pix pour le Gem, Polock pour le peintre Jackson !
Voila , je travaille actuellement sur un projet de "web cam painting" avec comme forme de peinceau des ".obj".
- je cherche des méthodes pour éventuellement à la fin de chaque création exporter automatiquement le résultat en "Jpg" voir au mieu stocker ce jpg directement sur un ftp.
- je cherche aussi à comprendre dans ce patch comment utiliser le multi_model , afin de changer alléatoirement "d'.obj" pour complexifier les créations.
- et dernière chose (la ça se complique) je souhaiterais qu'à la fin de chaque créations, que l'on puisse admirer le travail dans un apperçu d'espace 3D. c'est à dire "figer" le résultat et proposer un traveling caméra en perspective XYZ par exemple.
ya t il des gens motivé pour m'aider, me donner des pistes, me dire si cela etre possible ?
Je joins à ce mail le patch ainsi qu'un .obj à coller dans le DATA du gem
Merci D'avance !
Ce travail à déja été exposé dans des vitrines de galleries, il se projette sur une toile blanche de peintre, et s'argumente autour de la non-interactivité , c'est a dire dépossèder le controle du "spect-acteur" dans dans une position de présence à distance.
(version du mail en anglais assez pitoyable)
hey all !
I working on a project with pure data and Gem, the name is Pix_polock and use a web cam for create a form and colors with mouvements. I join my patch attach on this mail. But I whant to create an other functions :
- I whant to use a "multi_model" for different form of painting - use à system of "Jpg print" on a folder after 'bang time' or export on a server or FTP for example - and a must ! after creating in time, watch the draw in 3D perspective.
any idéas ? Thanks :)
On Thu, 14 Jun 2007, boris.vol@free.fr wrote:
voici le pix_polock !
ça serait mieux si ça s'appelait pix_pollock à la place.
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801, Montréal QC Canada
Hello Boris,
Note its "Pollock" not "polock"
.b.
boris.vol@free.fr wrote:
(french and english)
Bonjour à tous !
voici le pix_polock ! pix pour le Gem, Polock pour le peintre Jackson !
Voila , je travaille actuellement sur un projet de "web cam painting" avec comme forme de peinceau des ".obj".
- je cherche des méthodes pour éventuellement à la fin de chaque création
exporter automatiquement le résultat en "Jpg" voir au mieu stocker ce jpg directement sur un ftp.
- je cherche aussi à comprendre dans ce patch comment utiliser le multi_model ,
afin de changer alléatoirement "d'.obj" pour complexifier les créations.
- et dernière chose (la ça se complique) je souhaiterais qu'à la fin de chaque
créations, que l'on puisse admirer le travail dans un apperçu d'espace 3D. c'est à dire "figer" le résultat et proposer un traveling caméra en perspective XYZ par exemple.
ya t il des gens motivé pour m'aider, me donner des pistes, me dire si cela etre possible ?
Je joins à ce mail le patch ainsi qu'un .obj à coller dans le DATA du gem
Merci D'avance !
Ce travail à déja été exposé dans des vitrines de galleries, il se projette sur une toile blanche de peintre, et s'argumente autour de la non-interactivité , c'est a dire dépossèder le controle du "spect-acteur" dans dans une position de présence à distance.
(version du mail en anglais assez pitoyable)
hey all !
I working on a project with pure data and Gem, the name is Pix_polock and use a web cam for create a form and colors with mouvements. I join my patch attach on this mail. But I whant to create an other functions :
- I whant to use a "multi_model" for different form of painting
- use à system of "Jpg print" on a folder after 'bang time' or export on a
server or FTP for example
- and a must ! after creating in time, watch the draw in 3D perspective.
any idéas ? Thanks :)
GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
On Wed, 13 Jun 2007, IOhannes m zmoelnig wrote:
I can maybe see the utility on Linux but what you propose makes no sense on OSX or Windows.
it makes sense on windows, as there are already 2 (or 3, if you count the old vfw/avi stuff) APIs (directshow, quicktime). on os-x things are a bit different, as there is basically just one API. nevertheless, the platform that will benefit most from it will be linux.
IMHO it should handle also live video input (cameras) and live video output (X11, Xv, with or without SHM, and then SDL, OSX, ...). If it handles at least V4L1 and V4L2, count me in for GridFlow support.
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801, Montréal QC Canada
Mathieu Bouchard wrote:
On Wed, 13 Jun 2007, IOhannes m zmoelnig wrote:
I can maybe see the utility on Linux but what you propose makes no sense on OSX or Windows.
it makes sense on windows, as there are already 2 (or 3, if you count the old vfw/avi stuff) APIs (directshow, quicktime). on os-x things are a bit different, as there is basically just one API. nevertheless, the platform that will benefit most from it will be linux.
IMHO it should handle also live video input (cameras) and live video output (X11, Xv, with or without SHM, and then SDL, OSX, ...). If it handles at least V4L1 and V4L2, count me in for GridFlow support.
yes definitely. i have forgotten the v4l1 and v4l2 stuff in my enumeration of "stream readers", most likely because this was just to obvious.
video-out modules are what i meant by "stream writers" and could include just anything that is able to produce video out (e.g. output the video via a specialist hardware that is NOT a gfx card); it just saved me some typing to only mention firewire instead of losing myself in details about my ****-branded video-card.
so i'll try again: the framework should cover:: + stream readers (sequential read access to video-streams; with "video-stream" i mean anything that emits a number of images but where i (as audience) do not have control of the time) + random access readers (like stream-readers; but this time with added control on the time-axis) + stream writers (writing videos sequentialls)
i don't thing that "random access writers" do make much sense.
no assumptions about the interfaces accessed via this framework should be made (hardware, software, web,...)
mfga.sdr IOhannes
mfg,.asdr IOhannes
On Thu, 14 Jun 2007, IOhannes m zmoelnig wrote:
i have forgotten the v4l1 and v4l2 stuff in my enumeration of "stream readers", most likely because this was just to obvious.
I don't think that it was obvious. E.g. the objects for handling movie files, web streaming, cameras, are all different classes, and actually, there are more classes than that (even just reading movie files is involving several different classes)
typing to only mention firewire
Did you mention firewire? (oops. have I skipped over that word?)
i don't thing that "random access writers" do make much sense.
for uncompressed data, it's rather easy, but it's not that often useful.
no assumptions about the interfaces accessed via this framework should be made (hardware, software, web,...)
Some assumptions have to be made, because any piece of code embodies assumptions... any piece of code *is* an embodiment of assumptions.
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801, Montréal QC Canada
OK, a little bit more detailed description of my work: I will implement a framework which will be used to load the different libraries as plugins. This has the advantage that the libraries does not have to be available at compiling time (it will be checked if they are available at runtime) and various plugins can be added later easily. This framework should then replace the existing implementation in pix_film and pix_video. So now I'm looking at the API's of the different libraries and will then implement the framework.
The libraries for which the framework is aimed to are: for file read/write: - libAVIFile - quicktime4linux - ffmpeg - DirectShow - QuickTime
for stream read/write: - V4L2 - DV - QuickTime - DirectShow
I will also look at mplayer and media player classic .
So if you know other libraries that would be nice to have or programs which have a similar framework please let me know.
thomas
Thomas Holzmann schrieb:
Hello,
I have been chosen for Google Summer of Code and am working on a video input-output framework for GEM. This framework should make it possible to load different video encoders/decoders at runtime so that there is no need for them to be available at compiling time. So if you have some suggestions for my project it would be nice to let me know.
Thomas
GEM-dev mailing list GEM-dev@iem.at http://lists.puredata.info/listinfo/gem-dev
Thomas Holzmann wrote:
Hello,
I have been chosen for Google Summer of Code and am working on a video input-output framework for GEM.
i will add thomas to the developers at sourceforge so he can check in his changes. however, all changes should go into a BRANCH (and not into the main-TRUNK), until the whole thing works out satisfactory.
i therefore created a branch called videoIO.
fmgas.dr IOhannes
-
B. Bogart -
boris.vol＠free.fr -
chris clepper -
IOhannes m zmoelnig -
marius schebella -
Mathieu Bouchard -
Thomas Holzmann