hello,
I'm planning to build some video encoding and decoding plugin based on NVENC (see [1]). NVENC works directly with GPU memory, so I'm wondering if the plugins API allows me to work directly with Gem texture ? And if not, is it possible (I mean : is it no too huge work) to update plugins API to work with texture besides pix_block ?
I could make 2 externals to work directly with texture for video encoding and decoding. But NVENC is not the only API to do so, (I'm thinking about HAP on Mac OS, see [2]) so maybe it's time to update API ?
Currently, no library used by Gem is using GPU for video coding/decoding but Quicktime on Mac OS X which is now deprecated. And I think it could be a nice feature.
Cheers
Antoine
[1] : https://developer.nvidia.com/nvidia-video-codec-sdk [2] : http://vdmx.vidvox.net/blog/hap -- do it yourself http://antoine.villeret.free.fr
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 2014-09-14 21:00, Antoine Villeret wrote:
hello,
I'm planning to build some video encoding and decoding plugin based on NVENC (see [1]). NVENC works directly with GPU memory, so I'm wondering if the plugins API allows me to work directly with Gem texture ?
no.
And if not, is it possible (I mean : is it no too huge work) to update plugins API to work with texture besides pix_block ?
hmm, i have thought about that in the past, and i think we should NOT add this capabilities to video/film, BUT...
- - i would like to have all (current) plugins (that is: video, film, image and model) to be totally openGL-agnostic. currently all plugins but "model" already adhere to this (apart from using some GL_ constants for colorspaces). "model" does not and this is the reason why model-loading is currently totally broken for the default multi-context builds!
- - i *also* think that there is a need to directly input *AND* output textures via plugins. my main motivation for this is actually "syphon"[1] (rather than GPU-based decoding). another motivation is, that in days long gone, [pix_movie] would have optimized paths (at least on OSX), that were faster than the current [pix_film]+[pix_texture] implementation.
SO: i think, we need an API for "texin" and "texout" (that's two different plugins!). i guess "video" is a good starting point (without the 'dialog' parts), for both. we have to take care about how to properly support multiple-contexts. (e.g. the texture ID could be different for each window). i'm not entirely sure about the best way to ensure this.
internally we could have a texinFILM plugin, that would make all film-plugins available as texture source (just like there is a filmIMAGE plugin, that allows to load images as 1-frame films).
[pix_movie] could then be switched to the texin API.
and we could have two new objects [tex_input] and [tex_output] as "native" clients of these plugins.
fgmadr IOhannes
I'm aware of Syphon but iirc there is already a pd externals to deal with. But I never tried it. Btw I'm looking for such a solution (I mean texture sharing between context and process) for Linux for a while.
Concerning the texin and texout API, I understand what you mean but I'm not sure to be the man to build such an API :-) but this could be great !
+ a
-- do it yourself http://antoine.villeret.free.fr
2014-09-15 10:01 GMT+02:00 IOhannes m zmoelnig zmoelnig@iem.at:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 2014-09-14 21:00, Antoine Villeret wrote:
hello,
I'm planning to build some video encoding and decoding plugin based on NVENC (see [1]). NVENC works directly with GPU memory, so I'm wondering if the plugins API allows me to work directly with Gem texture ?
no.
And if not, is it possible (I mean : is it no too huge work) to update plugins API to work with texture besides pix_block ?
hmm, i have thought about that in the past, and i think we should NOT add this capabilities to video/film, BUT...
- i would like to have all (current) plugins (that is: video, film,
image and model) to be totally openGL-agnostic. currently all plugins but "model" already adhere to this (apart from using some GL_ constants for colorspaces). "model" does not and this is the reason why model-loading is currently totally broken for the default multi-context builds!
- i *also* think that there is a need to directly input *AND* output
textures via plugins. my main motivation for this is actually "syphon"[1] (rather than GPU-based decoding). another motivation is, that in days long gone, [pix_movie] would have optimized paths (at least on OSX), that were faster than the current [pix_film]+[pix_texture] implementation.
SO: i think, we need an API for "texin" and "texout" (that's two different plugins!). i guess "video" is a good starting point (without the 'dialog' parts), for both. we have to take care about how to properly support multiple-contexts. (e.g. the texture ID could be different for each window). i'm not entirely sure about the best way to ensure this.
internally we could have a texinFILM plugin, that would make all film-plugins available as texture source (just like there is a filmIMAGE plugin, that allows to load images as 1-frame films).
[pix_movie] could then be switched to the texin API.
and we could have two new objects [tex_input] and [tex_output] as "native" clients of these plugins.
fgmadr IOhannes
[1] http://syphon.v002.info/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1
iQIcBAEBCAAGBQJUFpzFAAoJELZQGcR/ejb4aisP/iiW9ssrx4eMsB3uNaLrlB7n KGhCK+25sMcnqDDBDbxcMvHoPADvTFEIA7Tta6en9YSpEr7xeaWRedp/2qb0uaYG 05AzWuDoAH4171cF8iwyq8LiiT5/6RjGpvpzhlR7jDYmjgRd7EDz8m6zRUM6u/CO sOpafnvK8EssQkeWO8cMPHXMzwCGdrhMjdoxL953doWgQOeXLDg3UbgYzs0JsRjG ilWrPtRRE05F5rX4/DTmV+ffrL0bVEyaFFeJ5cenDszhN+xY0KhlQeg3RuEVB9tB om7IjngeEKeTRlwHX6GeSaA+H/Mf5pDf14kgfnL49qkgMUDmdRvp/ahFyz/csFPF MERd/QgfwYYvc0J2I7Q5GEPebgPOl4BtyHQreq1y8xMDbGkFs+KqY15jk5Xt4j0N mQNiqr51UtQR5lhPpvjwcPEGADgulcTDH4f1J6o0/00Hz933rpwikxwL1BZH8Scc J4043FEorZvCAkJnBS0RaFBVCeGZsNcKtp1MV8nyBCyKaZrYh7GY7onp5bYOmAgh WEUR3VfY7CjhxLzIHCER7oLYAFc5zPlA5Gfz4IJInZh1MXcJeuWLmjzyr2FvOrOg escx69CTprsB3Qv699KD/73qbMXoyjS3Wh7x22pJxvN0QTwbkgQVvnZNoLDHi9pD eKBfqvlgBCuP4gVTfTeu =QvhL -----END PGP SIGNATURE-----
GEM-dev mailing list GEM-dev@lists.iem.at http://lists.puredata.info/listinfo/gem-dev
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
On 2014-09-15 10:55, Antoine Villeret wrote:
I'm aware of Syphon but iirc there is already a pd externals to deal with. But I never tried it.
i know that there are syphon externals. but i strongly dislike the idea of having a direct library dependency within the patch. (that's one of the main ideas behind the plugin foo).
Btw I'm looking for such a solution (I mean texture sharing between context and process) for Linux for a while.
afaik, syphon is still the only infrastructure that does this, and it's still OSX only. nevertheless one might come up in the future :-)
fgmsdr IOhannes
-
Antoine Villeret -
IOhannes m zmoelnig