Hi all, here is the first release of a simple python script object.
It does the following:
- you can load python scripts as modules and execute the functions therein
- different objects share modules, hence they should be able to cooperate
(not tested)
- the object has two inlets (one for the object control, one for arguments
to python) and one outlet for results from python
- it passes floats, symbols, lists, anythings to python as tuples and for
the results it processes python tuples, lists, floats, ints, strings.
- it should be good for list, string and number processing of any kind
- you can edit the scripts on-line and reload them anytime
Here are the caveats:
- the standard python distribution (2.2) does not provide a shared library,
hence the external will be statically linked. It contains the whole python interpreter and is therefore extremely bloated. (You have to make your Linux external yourself). I am very grateful for any hints how to dynamically link to Python.
- with MaxMSP it doesn't work at all - it loads but Max dies with the first
call to the Python API.
Future directions:
- py will remain as a simple 1-in/1-out solution executing python functions
- pyext will orthogonally implement a real object-oriented pd/python
external with multiple inlets and outlets and method processing
- i will try to make both objects execute on their own thread (switchable),
and provide hookpoints for other PD externals to extend the python functionality.
I found a severe bug in flext, therefore i uploaded a new version (0.2.1). The bug only affects implementations with more than one inlet processing non-atomic messages - like the py external.
you'll find all the stuff at the usual place: http://www.parasitaere-kapazitaeten.net/Pd/ext
greetings, Thomas
No problems compiling it, but when I load test.pd I get
ImportError: No module named scr2 ImportError: No module named scr1 ImportError: No module named scr1 ImportError: No module named scr2 ImportError: No module named scr2
This also happens if I move test.pd into the same directory as the scripts. I'm using python2.2 2.2-5 from Debian unstable.
On Sat, Mar 30, 2002 at 02:51:29AM +0100, Thomas Grill wrote:
Hi all, here is the first release of a simple python script object.
It does the following:
- you can load python scripts as modules and execute the functions therein
- different objects share modules, hence they should be able to cooperate
(not tested)
- the object has two inlets (one for the object control, one for arguments
to python) and one outlet for results from python
- it passes floats, symbols, lists, anythings to python as tuples and for
the results it processes python tuples, lists, floats, ints, strings.
- it should be good for list, string and number processing of any kind
- you can edit the scripts on-line and reload them anytime
Here are the caveats:
- the standard python distribution (2.2) does not provide a shared library,
hence the external will be statically linked. It contains the whole python interpreter and is therefore extremely bloated. (You have to make your Linux external yourself). I am very grateful for any hints how to dynamically link to Python.
- with MaxMSP it doesn't work at all - it loads but Max dies with the first
call to the Python API.
Future directions:
- py will remain as a simple 1-in/1-out solution executing python functions
- pyext will orthogonally implement a real object-oriented pd/python
external with multiple inlets and outlets and method processing
- i will try to make both objects execute on their own thread (switchable),
and provide hookpoints for other PD externals to extend the python functionality.
I found a severe bug in flext, therefore i uploaded a new version (0.2.1). The bug only affects implementations with more than one inlet processing non-atomic messages - like the py external.
you'll find all the stuff at the usual place: http://www.parasitaere-kapazitaeten.net/Pd/ext
greetings, Thomas
am 30.03.2002 4:24 Uhr schrieb Ben Saylor unter bsaylor@macalester.edu:
No problems compiling it, but when I load test.pd I get
ImportError: No module named scr2 ImportError: No module named scr1 ImportError: No module named scr1 ImportError: No module named scr2 ImportError: No module named scr2
py will search for the scripts in the pd path (-path option). Did you specify it to point to the script directory?
greetings, Thomas
Ah, got it. It works nicely. :)
There's one problem, though - changing a global variable in one instance changes it for all others of the same script. That makes it hard for each instance to have its own state variables (I'm not even sure how it would be done). In case I'm not making sense (it's very late ;) take a look at the attached patch and script.
Also, a minor bug: if a py object hasn't been given a function name yet and you send something to its right inlet, pd crashes.
Thanks, Ben
On Sat, Mar 30, 2002 at 09:44:11AM +0100, Thomas Grill wrote:
am 30.03.2002 4:24 Uhr schrieb Ben Saylor unter bsaylor@macalester.edu:
No problems compiling it, but when I load test.pd I get
ImportError: No module named scr2 ImportError: No module named scr1 ImportError: No module named scr1 ImportError: No module named scr2 ImportError: No module named scr2
py will search for the scripts in the pd path (-path option). Did you specify it to point to the script directory?
greetings, Thomas
There's one problem, though - changing a global variable in one instance changes it for all others of the same script. That makes it hard for each instance to have its own state variables (I'm not even sure how it would be done). In case I'm not making sense (it's very late ;) take a look at the attached patch and script.
Yes, a script is shared among all objects executing the same script. This is because there is only one instance of the Python interpreter and the script is loaded as a module (once). So, if you want to have different instances, you have to use different scripts.
With pyext (the real Python external) the situation will be a bit clearer.
Also, a minor bug: if a py object hasn't been given a function name yet and you send something to its right inlet, pd crashes.
Thanks, i'll fix that.
greetings, Thomas
I worry a lot about this external, as well as about Joe Sarlo's GrIPD. Python is not anywhere near a real time language. Run time environments with automatic memory deallocation are very tricky to get to work in bounded time. I worry that when a moderate to large amount of Python code gets loaded, and the program runs for a while, its garbage collection and paging behavior will cause timing failures in even simple MIDI processing. Am I just being up tight about all music code being bounded time, or does anyone else see this as a real issue?
On Fri, 2002-03-29 at 20:51, Thomas Grill wrote:
here is the first release of a simple python script object.
am 30.03.2002 4:34 Uhr schrieb jfm3 unter jfm3@ouroboros-complex.org:
I worry a lot about this external, as well as about Joe Sarlo's GrIPD. Python is not anywhere near a real time language. Run time environments with automatic memory deallocation are very tricky to get to work in bounded time. I worry that when a moderate to large amount of Python code gets loaded, and the program runs for a while, its garbage collection and paging behavior will cause timing failures in even simple MIDI processing. Am I just being up tight about all music code being bounded time, or does anyone else see this as a real issue?
Hi, it is very likely that you are right worrying about real-time-capability of python. (you can turn off garbage collection, btw.) I will try to elaborate a bit more on this issue. On the other hand my primary goal for the next time is to extend pd and max in the direction of non-time (buffer-oriented) features, with spectral and time-paradox functions. For that, python provides the most versatile way for algorithmic compsition. Watch out for the VASP (vector assembling signal processor).
greetings, Thomas
i wonder how the jMax people deal with this as their gui is written in Java? does anyone know?
On 29 Mar 2002, jfm3 wrote:
I worry a lot about this external, as well as about Joe Sarlo's GrIPD. Python is not anywhere near a real time language. Run time environments with automatic memory deallocation are very tricky to get to work in bounded time. I worry that when a moderate to large amount of Python code gets loaded, and the program runs for a while, its garbage collection and paging behavior will cause timing failures in even simple MIDI processing. Am I just being up tight about all music code being bounded time, or does anyone else see this as a real issue?
On Fri, 2002-03-29 at 20:51, Thomas Grill wrote:
here is the first release of a simple python script object.
-- (jfm3)
Josh .. Yoshi .. Joschi .. xiphoidprocess.com .. eds.org/~joschi
Thomas Grill wrote:
i wonder how the jMax people deal with this as their gui is written in Java? does anyone know?
It _is_ slow and jMax is not nearly as stable as pd.
Thomas
And, presumably, the gui is in a seperate thread (?)
Larry
On Sat, 30 Mar 2002, Larry Troxler wrote:
Thomas Grill wrote:
i wonder how the jMax people deal with this as their gui is written in Java? does anyone know?
It _is_ slow and jMax is not nearly as stable as pd.
Thomas
And, presumably, the gui is in a seperate thread (?)
It runs in a completely separated *process* in fact.
Maurizio Umberto Puxeddu.
On Sat, 2002-03-30 at 04:23, Josh Steiner wrote:
i wonder how the jMax people deal with this as their gui is written in Java? does anyone know?
The same way PD deals with Tcl/TK - by sending messages over a socket. This could be described as threading with processes :)
Karl
On 29 Mar 2002, jfm3 wrote:
I worry a lot about this external, as well as about Joe Sarlo's GrIPD. Python is not anywhere near a real time language. Run time environments with automatic memory deallocation are very tricky to get to work in bounded time. I worry that when a moderate to large amount of Python code gets loaded, and the program runs for a while, its garbage collection and paging behavior will cause timing failures in even simple MIDI processing. Am I just being up tight about all music code being bounded time, or does anyone else see this as a real issue?
On Fri, 2002-03-29 at 20:51, Thomas Grill wrote:
here is the first release of a simple python script object.
-- (jfm3)
Josh .. Yoshi .. Joschi .. xiphoidprocess.com .. eds.org/~joschi
Computer Music Department
Peabody Institute of the Johns Hopkins University karlmac@peabody.jhu.edu
mambo.peabody.jhu.edu/~karlmac
jfm3 wrote:
I worry a lot about this external, as well as about Joe Sarlo's GrIPD.
And don't forget my scheme extern http:/www.westnet.com/~lt/pd
Python is not anywhere near a real time language. Run time environments with automatic memory deallocation are very tricky to get to work in bounded time. I worry that when a moderate to large amount of Python code gets loaded, and the program runs for a while, its garbage collection and paging behavior will cause timing failures in even simple MIDI processing. Am I just being up tight about all music code being bounded time, or does anyone else see this as a real issue?
I can't answer for Python, but I assume that any garbage-collecting language engine would allow you to disable automatic garbage collection, so that it doesn't happen at unexpected times behind your back.
So, it becomes an issue of whether you have enough ram to cover you for the duration of your performance.
When working with my scheme external, I rationalized the situation thusly: well, I certainly want to stream any performance I do with PD to disk, right? So I have to make sure my machine has enough disk space to stream the duration of the performance. Likewise, I have this scheme extern that eats up RAM as time goes on, so I should make sure my machine has enough RAM to cover the duration of the performance.
Larry
On Fri, 2002-03-29 at 20:51, Thomas Grill wrote:
here is the first release of a simple python script object.
-- (jfm3)
Name: signature.ascsignature.asc Type: application/pgp-signature Description: This is a digitally signed message part
On Sat, 30 Mar 2002, Larry Troxler wrote:
jfm3 wrote:
I worry a lot about this external, as well as about Joe Sarlo's GrIPD.
And don't forget my scheme extern http:/www.westnet.com/~lt/pd
Python is not anywhere near a real time language. Run time environments with automatic memory deallocation are very tricky to get to work in bounded time. I worry that when a moderate to large amount of Python code gets loaded, and the program runs for a while, its garbage collection and paging behavior will cause timing failures in even simple MIDI processing. Am I just being up tight about all music code being bounded time, or does anyone else see this as a real issue?
I can't answer for Python, but I assume that any garbage-collecting language engine would allow you to disable automatic garbage collection, so that it doesn't happen at unexpected times behind your back.
I'm interested in this thread because I embedded a Python interpreter in Csound few days ago. No release, but I using it. It is documented here:
http://pythonsound.sf.net/pycsound.pdf
I know that Python is not a real-time language. RT is not a must for me at the moment but obviously people is interested in a possible use of PyCsound in RT.
The situation of memory management in Python is particular. Here is how *I* understand the situation. I write so that people can confirm or deny what I say.
Python has 2 different methods of GC: reference count and cyclic reference detection. Reference count is performed in a well defined amount of time and it is the default GC method. It works as far as you don't make cyclic references. The second GC method has been added to fix this problem and it periodically scans the list of object, searching for cyclic reference. When possible it destroy them. This method is optional and can be disabled and you can check how often it has to be performed. Of course more objects you have, longer it will take to perform the check (not RT-friendly). So if you disables the optional GC and write your code so that you make no reference cycles, you should be able to do long performances.
Maurizio Umberto Puxeddu.
i wonder how the jMax people deal with this as their gui is written in Java? does anyone know?
It runs in a completely separated *process* in fact.
It is my goal to let python scripts run on a separated thread as well, but for that i need to pack a few more things into flext (message queue, thread safety,...).
In that context: Which pd calls are allowed in a second thread? clock_delay(), outlet_*() would be the most important
greetings, Thomas
Thomas Grill schrieb:
In that context: Which pd calls are allowed in a second thread? clock_delay(), outlet_*() would be the most important
outlet_*() works on Win but crashes on Linux.... (or at least displays a lot of 'stack overflow' messages in the console window).
Olaf
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Sat, 30 Mar 2002, Olaf Matthes wrote:
::outlet_*() works on Win but crashes on Linux.... (or at least displays a lot of ::'stack overflow' messages in the console window).
had the same problem with a corba plugin. increasing the stacksize in m_obj.c (around line 258) helped:
- -#define STACKSIZE 1000000
+#define STACKSIZE 50000000
lu
I have been using Python directly as the glue language in a DSP environment for some time now (as well as using Joe's GriPD) and have had no noticeable timing issues. I do, however, use Python 2.0 which does not include a garbage collector, only reference counting, which IMHO is more predictable. However, Python is designed such that as long as you don't do anything wacky with "pointers" such as cyclical structures, the garbage collector should never even kick in for a noticeable amount of time.
In general, I think it's more useful to try these kinds of things IN CONTEXT before worrying about their apparent limitations, because of the advantages of moving to such a flexible system can be pretty great.
On Fri, 29 Mar 2002, jfm3 wrote:
I worry a lot about this external, as well as about Joe Sarlo's GrIPD. Python is not anywhere near a real time language. Run time environments with automatic memory deallocation are very tricky to get to work in bounded time. I worry that when a moderate to large amount of Python code gets loaded, and the program runs for a while, its garbage collection and paging behavior will cause timing failures in even simple MIDI processing. Am I just being up tight about all music code being bounded time, or does anyone else see this as a real issue?
On Fri, 2002-03-29 at 20:51, Thomas Grill wrote:
here is the first release of a simple python script object.
I cannot comment on Thomas Grill's python object since I haven't looked at it yet, but GC is not really an issue for GrIPD since all of the python code runs in a separate process, and on Linux, at the lowest priority (SCHED_OTHER) when launched from PD. So, and correct me if I'm wrong, if you run PD with -rt, nothing in the GrIPD python process should directly effect audio calculation (other than memory utilization). The worst that should happen would be small glitches in GUI response and updates, and the very worst case scenario would be the GUI becoming unresponsive, but there's really no way to avoid that issue. The PD tcl/tk GUI has the same behavior.
Joe
| | Joseph A. Sarlo | | jsarlo@mambo.peabody.jhu.edu |______________________________
On 29 Mar 2002, jfm3 wrote:
I worry a lot about this external, as well as about Joe Sarlo's GrIPD. Python is not anywhere near a real time language. Run time environments with automatic memory deallocation are very tricky to get to work in bounded time. I worry that when a moderate to large amount of Python code gets loaded, and the program runs for a while, its garbage collection and paging behavior will cause timing failures in even simple MIDI processing. Am I just being up tight about all music code being bounded time, or does anyone else see this as a real issue?
On Fri, 2002-03-29 at 20:51, Thomas Grill wrote:
here is the first release of a simple python script object.
This is definitely an issue, but it is one that already is a problem for PD. The most obvious example is that data is written to a socket to drive the current PD gui. Those calls to write on the socket are not garunteed to complete in bounded time any more than the Python interpreter is (in fact, I would argue that there is more of a chance of the socket blocking than the Python gc interrupting things). I think that this external is likely to work just like PD does now - pretty well for most cases. To get something that works well all of the time is going to require multi-threading and the ability for externals to easily spawn background threads that communicate with the main message passing system asynchrously. There are a lot of Dannenberg papers on these issues - I can get specific references if you want.
Karl
On Fri, 2002-03-29 at 22:34, jfm3 wrote:
I worry a lot about this external, as well as about Joe Sarlo's GrIPD. Python is not anywhere near a real time language. Run time environments with automatic memory deallocation are very tricky to get to work in bounded time. I worry that when a moderate to large amount of Python code gets loaded, and the program runs for a while, its garbage collection and paging behavior will cause timing failures in even simple MIDI processing. Am I just being up tight about all music code being bounded time, or does anyone else see this as a real issue?
On Fri, 2002-03-29 at 20:51, Thomas Grill wrote:
here is the first release of a simple python script object.
-- (jfm3)
Computer Music Department
Peabody Institute of the Johns Hopkins University karlmac@peabody.jhu.edu
mambo.peabody.jhu.edu/~karlmac
-
Ben Saylor -
jfm3 -
Joseph A. Sarlo -
Josh Steiner -
Karl MacMillan -
Larry Troxler -
ludwig zeininger -
Maurizio Umberto Puxeddu -
Michael Droettboom -
Olaf Matthes -
Thomas Grill