Hi all, i'm thinking about the future of VASP these days since i'm myself using it for compositional work. Said this, it is not completely true though, since i'm using the numarray extension of the py/pyext external (CVS version), which allows to work with PD arrays in Python. I consider this the future of VASP (namely to be a pure Python module), although it will perhaps be called differently then. Realizing there are still a number of daily downloads although i haven't touched VASP for quite a while now let's me ask you about your usage of VASP. Do you use it for
- simple array math/transformation?
- medium complex things like analysis or synthesis?
- complex work like algorithmic composition and the likes?
- none of the above?
- something completely different?
Does it have to provide
- real-time response (like for interaction with DSP processing)?
- or not?
Do you predominantly use
- singular VASP objects?
- groups/whole subpatchers of interacting VASP objects?
I realized in my own work that it's very hard to express complex stuff as PD graphs, and also VASP has the problem of needing to lock arrays while working on them, which still isn't really implemented in PD (it is in the devel branch, but the future of this isn't clear either). The question is now whether i abandon the current PD object implementation of VASP and focus on the Python one, or think about a side-by-side solution of both systems, probably reducing the functionality of the current VASP implementation a bit. There's still time to decide since the Python array implementation hasn't been standardized yet.... Let me know what you think and if you care at all.
best greetings, Thomas
I don't use VASP much, because of some of its shortcomings. If those (see below) could be fixed I would learn how to do back flips just so I could express how happy I would be.
Thomas Grill wrote:
VASP. Do you use it for
- simple array math/transformation?
yes
- medium complex things like analysis or synthesis?
- complex work like algorithmic composition and the likes?
yes
- none of the above?
- something completely different?
Does it have to provide
- real-time response (like for interaction with DSP processing)?
yes!
- or not?
Do you predominantly use
- singular VASP objects?
- groups/whole subpatchers of interacting VASP objects?
usually small bunches of 3-8 VASP objects. a few bigger abstractions
I realized in my own work that it's very hard to express complex stuff as PD graphs, and also VASP has the problem of needing to lock arrays while working on them, which still isn't really implemented in PD (it is in the devel branch, but the future of this isn't clear either). The question is now whether i abandon the current PD object implementation of VASP and focus on the Python one, or think about a side-by-side solution of both systems, probably reducing the functionality of the current VASP implementation a bit. There's still time to decide since the Python array implementation hasn't been standardized yet.... Let me know what you think and if you care at all.
best greetings, Thomas
PD-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://iem.at/cgi-bin/mailman/listinfo/pd-list
I read:
I would learn how to do back flips just so I could express how happy I would be.
if you happen to be near zurich or vienna at some point I can teach you
take care,
x
At 13:33 23.03.2005, you wrote:
I read:
I would learn how to do back flips just so I could express how happy I would be.
if you happen to be near zurich or vienna at some point I can teach you
yea, spring + sommer is coming and you gotta show me the new tricks you've practiced in the schwimmhalle this winter. are you ready for the big bridge???
hey thomas, since i've discovered numpy i use vasp just to get audio into python (evil hack and lot's of overhead) to process it with numpy. anything that's possible with vasp and much more is possible with numpy and numpy is really very optimised. even if one is not a python programmer learning basic python and numpy shouldn't take much longer than learning how to use vasp. and learning python is something you'll never regret anyway... imho> if it was possible with pyext to get audio into python and back and if threading in pyext was more stable, there wouldn't be much need for vasp (assumed that one is willing to learn basic python). AND not to be forgotten, numpy is also great for imageprocessing...... . . . ! ! ! !! !! !!! !!!!!!!!!
sven.
Hey Sven, the current cvs py/pyext version does exactly what you want, except that it doesn't use numpy but rather numarray. There is currently a debate going on about what the future of arrays in Python might be, and there's a new project called numeric3 which shall once unite the strengths of both numeric and numarray. Numeric3 is said to have good chances to go into the Python core. py/pyext supports numarray for buffer and signal processing - i haven't released it yet as i heard there have been crashes with the dsp part and at the moment i don't have time to test that. buffer processing is quite efficient as PD buffers are directly mapped to Python arrays, signal processing is quite slow on the other hand because of the overhead of calling into Python code. Using psyco JIT compilation helps a bit though. A large part of current VASP functionality is already covered by the numarray functions, and i plan to bundle the other, more music-specific things into a VASP module, using mainly Python code and - if needed - the C-extensibility of numarray. Please note that i chose numarray over Numeric only because there was a recommendation to do so, which has changed in the meantime - i'm ready to jump on the line that leads to standardization. Summarizing, i would recommend that you check out the cvs versions of flext and py/pyext, install the numarray and psyco extensions to Python, and try out the new features (there are example patches for buffer and signal processing). I'm convinced that threading has become _quite_ stable in the meantime - if there are still issues please report back in detail, so that i can fix them. Oh yes, clearly i'd like to integrate image processing some day - although i have no idea yet, when this will happen. I'd really like to have a PDP/GEM-py/pyext bridge, but again, Python array support is not stable yet, so there's no need to hurry.
best greetings, Thomas
Am 23.03.2005 um 00:52 schrieb sven:
hey thomas, since i've discovered numpy i use vasp just to get audio into python (evil hack and lot's of overhead) to process it with numpy. anything that's possible with vasp and much more is possible with numpy and numpy is really very optimised. even if one is not a python programmer learning basic python and numpy shouldn't take much longer than learning how to use vasp. and learning python is something you'll never regret anyway... imho> if it was possible with pyext to get audio into python and back and if threading in pyext was more stable, there wouldn't be much need for vasp (assumed that one is willing to learn basic python). AND not to be forgotten, numpy is also great for imageprocessing...... . . . ! ! ! !! !! !!! !!!!!!!!!
sven.
PD-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://iem.at/cgi-bin/mailman/listinfo/pd-list
ok i'll check the cvs. numarray and numpy co-exist quite peacefully anyway, on of the reasons why i use numpy is scipy which relies on numpy. it already contains modules for signal and image processing and i'm fiddling around with its genetic algorithms which are great. i wouldn't say that signal processing is slow because most of what you want can be done with array operations, so there's actually not that much python needed if you're following the matlab paradigm of problem solving. when i find some time i'll try to boil down what causes the threading crashes. the problem is that it crashes after one minute or maybe after one hour (without any error messages) so the cause of the crash is quite hard to track down. anyway - thanks for the good work and i'm happy that you're continuing to work on pyext which is my favourite pd external...
sven.
At 11:01 23.03.2005, Thomas Grill wrote:
Hey Sven, the current cvs py/pyext version does exactly what you want, except that it doesn't use numpy but rather numarray. There is currently a debate going on about what the future of arrays in Python might be, and there's a new project called numeric3 which shall once unite the strengths of both numeric and numarray. Numeric3 is said to have good chances to go into the Python core. py/pyext supports numarray for buffer and signal processing - i haven't released it yet as i heard there have been crashes with the dsp part and at the moment i don't have time to test that. buffer processing is quite efficient as PD buffers are directly mapped to Python arrays, signal processing is quite slow on the other hand because of the overhead of calling into Python code. Using psyco JIT compilation helps a bit though. A large part of current VASP functionality is already covered by the numarray functions, and i plan to bundle the other, more music-specific things into a VASP module, using mainly Python code and - if needed - the C-extensibility of numarray. Please note that i chose numarray over Numeric only because there was a recommendation to do so, which has changed in the meantime - i'm ready to jump on the line that leads to standardization. Summarizing, i would recommend that you check out the cvs versions of flext and py/pyext, install the numarray and psyco extensions to Python, and try out the new features (there are example patches for buffer and signal processing). I'm convinced that threading has become _quite_ stable in the meantime - if there are still issues please report back in detail, so that i can fix them. Oh yes, clearly i'd like to integrate image processing some day - although i have no idea yet, when this will happen. I'd really like to have a PDP/GEM-py/pyext bridge, but again, Python array support is not stable yet, so there's no need to hurry.
best greetings, Thomas
Hi Sven, i was looking into making py/pyext conform to the old-style Numeric API rather than numarray, but it's hard to do that because of missing notification when a directly mapped buffer gets dereferenced. numarray implements that cleanly, using the Python buffer protocol. I think i'll wait for Numeric3 to become Numeric-compatible.
best greetings, Thomas
sven schrieb:
ok i'll check the cvs. numarray and numpy co-exist quite peacefully anyway, on of the reasons why i use numpy is scipy which relies on numpy. it already contains modules for signal and image processing and i'm fiddling around with its genetic algorithms which are great.
hi thomas,
thanks for your work, and bringing future vasp direction to the forum, and yes, i care!
1> usage:
vasp is a critical addition to most of my pd compositions these days, i use it essentially for sample based analysis (zero-crossings, peaks, density, etc) and minor editing and transformations of arrays (normalize, crop, pitch shift, enveloping, etc).
i generally have subpatches of analytical vasp objects for each array, which operate on soundfiler action or transformation change, and output the results as lists, which i use for algorythmic sequencing, and interpolation information (ie. morphs)
the transformation objects are maninly used for multipurpose sample editing, however i do generally normalize samples on array change. i find being able to tweak and edit samples non-rt within pd is an absolute luxury, and although pd has issues with lengthy arrays, and is missing some graphical objects that would make editing simpler (i'm working on it), i think this an important extention to pd's rt focus.
2> real-time response
i prefer vasp as a background processor, so i'm not concerned by real-time responsiveness, however this may change if i begin to use vasp differently.
3> object structure
i like the current multi-object aproach of vasp and have found it intuitive on learning (the documentation is great, thanks) as it gently mirrors pd's rt object model. i'm not sure how you plan to implement the python model, so i don't think i'd be prepared to make a comment on which is better. certainly the multi-object model works more seamlessly into the pd programming logic, although it may not be the most efficient way of expressing non-rt events. i think perhaps if i could express vasp routines in a single scriptlet, i would use it that way, but i still believe that for vasp to be intuitive to new users in pd it needs singular objects.
so, could you please outline your ideas for the python based vasp so that i can better understand how the app would function.
hmm.. i think that sorta sums me up!
ciao..
dmotd
At 04:56 23.03.2005, dmotd wrote:
think perhaps if i could express vasp routines in a single scriptlet, i would use it that way, but i still believe that for vasp to be intuitive to new users in pd it needs singular objects.
so, could you please outline your ideas for the python based vasp so that i can better understand how the app would function.
scriplets, that's exactly what you would do in python. that hasn't much to do with vasp anymore. pyext would be the bridge between pd and python, the logic and computation is beeing done in python using numarray then. check http://stsdas.stsci.edu/numarray/Doc/ for numarray docs
sven.
sven wrote:
scriplets, that's exactly what you would do in python. that hasn't much to do with vasp anymore. pyext would be the bridge between pd and python, the logic and computation is beeing done in python using numarray then. check http://stsdas.stsci.edu/numarray/Doc/ for numarray docs
sven.
yes i see, as i thought, thanks for the docs! i am a little concerned that vasp would be merged into pyext, not because python is a bad idea, but simply because losing the high-level object/abstraction approach could lose users with it.. after all there must be good reason why some pd users prefer pd's structure to say supercollider/csound..
this is of course not to say that bridging languages with pd is a bad idea, there are obvious advantages to doing so. however i think is is bad practice to expect new users, especially those without previous programming experience, to learn a new language/syntax on top of learning pd, not to mention the added dependencies that this brings.
as i said in my previous post, non-rt processing to me is a critical addition to the rt focus of pd, i'm not sure that others would agree with me, or if vasp is even known to new users (it rarely gets a mention on pd-list) but i think it would be sad to lose the imediacy of the concept behind the facade of pyext, which to me suggests something else, albeit potentially the same or greater.
anyhow, i hope i haven't missed the point or wandered too OT..
ciao..
dmotd
hi thomas,
I read:
numarray extension of the py/pyext external (CVS version), which allows to work with PD arrays in Python. I consider this the future of VASP
actually I find this an execellent idea, and it would open all sorts of other possibilities (think pyGSL, csound style GEN routines etc...)
Realizing there are still a number of daily downloads although i haven't touched VASP for quite a while now let's me ask you about your usage of VASP. Do you use it for
well, if VASP should significantly change it wouldn't break much of my existing patches, those could still be simply converted to VASp_OLD with a tiny perl/sed/tr/py script
- simple array math/transformation?
yes
- medium complex things like analysis or synthesis?
yes (though rather simple synthesis mostly)
- complex work like algorithmic composition and the likes?
nope / not yet, (I find supercollider more expressive for such things)
- something completely different?
yes but not interesting enough to bother
Does it have to provide
- real-time response (like for interaction with DSP processing)?
since I always thought it was nrt I virtuously used it that way but it would of course be nice to have this feature.
Do you predominantly use
- singular VASP objects?
rather
- groups/whole subpatchers of interacting VASP objects?
sometimes
Let me know what you think and if you care at all.
we care baby :)
cu
x
hi Thomas,
Thomas Grill wrote: ...
I realized in my own work that it's very hard to express complex stuff as PD graphs, and also VASP has the problem of needing to lock arrays while working on them, which still isn't really implemented in PD (it is in the devel branch, but the future of this isn't clear either). The question is now whether i abandon the current PD object implementation of VASP and focus on the Python one, or think about a side-by-side solution of both systems, probably reducing the functionality of the current VASP implementation a bit. There's still
have you been thinking about replacing vasp with "pyobj" (and possibly "pyarr") python externals, which would allow passing references to python objects inside pd messages?
This means wrapping PyObject* and PyArrayObject* inside the t_atom structure. It is generally possible, although, currently, not very elegant, and not 100% robust. In the "plustot" experiment, I found out, that until things change on the t_atom scene, the type best suited for extending is t_symbol. I have faked it into becoming a reference-counted Tcl_Obj wrapper, which is fool-proof in most cases, but may fail, of course, and crash Pd, if someone renames an array to such a faked symbol, etc.
Perhaps, with some discussion and lobbying, we would have an appropriate Pd atom type some day...
Krzysztof
Krzysztof Czaja wrote: ...
suited for extending is t_symbol. I have faked it into becoming a reference-counted Tcl_Obj wrapper, which is fool-proof in most cases, but may fail, of course, and crash Pd, if someone renames an array to such a faked symbol, etc.
this is misleading, sorry. The main failure of plustot is when foreign objects store faked symbols beyond lifetime, of course.
Krzysztof
Hi Krzysztof, using the PyObject pointers wrapped into t_symbols is really a great idea (i haven't really thought of that in such a direct form). I'm cautious however not to overload the symbol table. The current VASP system is able to use "immediate" vasps [vasp.!], that are buffer fragments independent from PD arrays, with reference counting, also wrapped into symbols - but these are reused if possible, so that the impact on the symbol table is negligible. I have to investigate whether it's feasible with two-level references to also reuse PD symbols for PyObject pointers. In general i think this is the way to go - to be able to have discrete VASP objects without having to duplicate code - a code-wise light system totally based on Python, which is (on par with C++) my favorite language, i admit. many thanks for the hint!
best greetings, Thomas
PS. I wonder how an appropriate t_atom type would look like - it seems like additionally to a void *pointer there should also be some namespace ID (the generator instance of the pointer) - that means a new structure like t_symbol and a respective pointer as a union member into t_atom. PPS. In this respect quite unrelated - i can't really express how i like Mathieu's idea of local symbol tables - i wish someone takes the time to implement a draft into devel_0_38 - to my mind this would be a big leap forwards
Am 23.03.2005 um 14:48 schrieb Krzysztof Czaja:
hi Thomas,
Thomas Grill wrote: ...
I realized in my own work that it's very hard to express complex stuff as PD graphs, and also VASP has the problem of needing to lock arrays while working on them, which still isn't really implemented in PD (it is in the devel branch, but the future of this isn't clear either). The question is now whether i abandon the current PD object implementation of VASP and focus on the Python one, or think about a side-by-side solution of both systems, probably reducing the functionality of the current VASP implementation a bit. There's still
have you been thinking about replacing vasp with "pyobj" (and possibly "pyarr") python externals, which would allow passing references to python objects inside pd messages?
This means wrapping PyObject* and PyArrayObject* inside the t_atom structure. It is generally possible, although, currently, not very elegant, and not 100% robust. In the "plustot" experiment, I found out, that until things change on the t_atom scene, the type best suited for extending is t_symbol. I have faked it into becoming a reference-counted Tcl_Obj wrapper, which is fool-proof in most cases, but may fail, of course, and crash Pd, if someone renames an array to such a faked symbol, etc.
Perhaps, with some discussion and lobbying, we would have an appropriate Pd atom type some day...
Krzysztof
PD-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://iem.at/cgi-bin/mailman/listinfo/pd-list
Oh well, somehow preempting Mathieu - i'm sure gridflow does it in a similar way - i'm sure he can elaborate on this quite a bit. Does it make sense to have two systems potentially providing the same features, only coming from two different directions? Ok, it would be interesting to do some benchmarks then ;-)
best, Thomas
Am 23.03.2005 um 23:48 schrieb Thomas Grill:
Hi Krzysztof, using the PyObject pointers wrapped into t_symbols is really a great idea (i haven't really thought of that in such a direct form). I'm cautious however not to overload the symbol table. The current VASP system is able to use "immediate" vasps [vasp.!], that are buffer fragments independent from PD arrays, with reference counting, also wrapped into symbols - but these are reused if possible, so that the impact on the symbol table is negligible. I have to investigate whether it's feasible with two-level references to also reuse PD symbols for PyObject pointers. In general i think this is the way to go - to be able to have discrete VASP objects without having to duplicate code - a code-wise light system totally based on Python, which is (on par with C++) my favorite language, i admit. many thanks for the hint!
best greetings, Thomas
PS. I wonder how an appropriate t_atom type would look like - it seems like additionally to a void *pointer there should also be some namespace ID (the generator instance of the pointer) - that means a new structure like t_symbol and a respective pointer as a union member into t_atom. PPS. In this respect quite unrelated - i can't really express how i like Mathieu's idea of local symbol tables - i wish someone takes the time to implement a draft into devel_0_38 - to my mind this would be a big leap forwards
Am 23.03.2005 um 14:48 schrieb Krzysztof Czaja:
hi Thomas,
Thomas Grill wrote: ...
I realized in my own work that it's very hard to express complex stuff as PD graphs, and also VASP has the problem of needing to lock arrays while working on them, which still isn't really implemented in PD (it is in the devel branch, but the future of this isn't clear either). The question is now whether i abandon the current PD object implementation of VASP and focus on the Python one, or think about a side-by-side solution of both systems, probably reducing the functionality of the current VASP implementation a bit. There's still
have you been thinking about replacing vasp with "pyobj" (and possibly "pyarr") python externals, which would allow passing references to python objects inside pd messages?
This means wrapping PyObject* and PyArrayObject* inside the t_atom structure. It is generally possible, although, currently, not very elegant, and not 100% robust. In the "plustot" experiment, I found out, that until things change on the t_atom scene, the type best suited for extending is t_symbol. I have faked it into becoming a reference-counted Tcl_Obj wrapper, which is fool-proof in most cases, but may fail, of course, and crash Pd, if someone renames an array to such a faked symbol, etc.
Perhaps, with some discussion and lobbying, we would have an appropriate Pd atom type some day...
Krzysztof
PD-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://iem.at/cgi-bin/mailman/listinfo/pd-list
PD-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://iem.at/cgi-bin/mailman/listinfo/pd-list
On Thu, 24 Mar 2005, Thomas Grill wrote:
Oh well, somehow preempting Mathieu - i'm sure gridflow does it in a similar way - i'm sure he can elaborate on this quite a bit. Does it make sense to have two systems potentially providing the same features, only coming from two different directions? Ok, it would be interesting to do some benchmarks then ;-)
GridFlow has several mechanisms for pointers.
GridFlow 0.7 transmits grids using a callback mechanism based on C++ function-pointers. Those are transmitted as a pair of floats that are casts of 16-bit integers that are the two halves of the pointer.
GridFlow 0.8 instead uses Pd's gpointers even though the pointee is not even of type t_pd. This is marginally faster, but also, it works in 64-bit mode.
GridFlow 0.8 also has a Pointer class on the ruby side for wrapping gpointers.
GridFlow 0.8 (and maybe late-0.7) uses unique receive-symbols for getting messages back from Tcl/Tk.
In any case, GridFlow *never* passes pointers to grid buffers in the Pd system, especially as most grids never need to be transmitted as one whole thing, and instead can be streamed between objects, in small chunks. This will change in the future, but the main mechanism will stay the same, I'll just add more options.
Mathieu Bouchard -=- Montréal QC Canada -=- http://artengine.ca/matju
On Wed, 23 Mar 2005, Thomas Grill wrote:
I have to investigate whether it's feasible with two-level references to also reuse PD symbols for PyObject pointers.
Yes because it (my system) is backwards-compatible, because I will only add new semantics, not breaking any, except if anyone uses the following syntax: dollar, argument number, period, symbol. (e.g. $0.hello). This syntax will be reserved for the new system. The rest will work just the same.
In addition, $0.hello will be 99% like the current $0-hello except that it will have to be supported explicitly by externals. It may take some time before externals adapt to the new way, and so, in most cases the $0-hello will have to stay for a while.
PS. I wonder how an appropriate t_atom type would look like - it seems like additionally to a void *pointer there should also be some namespace ID (the generator instance of the pointer) - that means a new structure like t_symbol and a respective pointer as a union member into t_atom.
There will not be a new t_word union member. However there will be a new atom type, *and* most bits in the a_type field will be reserved for the namespace id, that is, the number of the abstraction instance, at least in the case of that particular atom-type.
Mathieu Bouchard -=- Montréal QC Canada -=- http://artengine.ca/matju
-
CK -
dmotd -
Ian Smith-Heisters -
Krzysztof Czaja -
Mathieu Bouchard -
sven -
Thomas Grill