Thomas posted an idea a while back about making all of the core Pd objects into a library and having Pd itself just be the very core. I am thinking of doing this for the next Pd-extended, and this got me thinking. It would be great to have a SIMD version of the Pd-core lib as well. This would be easier to deploy and maintain since it could be in /externals/ rather than part of Pd.
Then to use it, you would change "-lib pd-core" to "-lib pd-core-simd" or whatever. How does this sound?
.hc
________________________________________________________________________ ____
"The arc of history bends towards justice." - Dr. Martin Luther King, Jr.
On Wed, 18 Jan 2006, Hans-Christoph Steiner wrote:
Thomas posted an idea a while back about making all of the core Pd objects into a library and having Pd itself just be the very core. I am thinking of doing this for the next Pd-extended, and this got me thinking. It would be great to have a SIMD version of the Pd-core lib as well. This would be easier to deploy and maintain since it could be in /externals/ rather than part of Pd. Then to use it, you would change "-lib pd-core" to "-lib pd-core-simd" or whatever. How does this sound?
I'm all for it.
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju | Freelance Digital Arts Engineer, Montréal QC Canada
Am 19.01.2006 um 09:12 schrieb Mathieu Bouchard:
On Wed, 18 Jan 2006, Hans-Christoph Steiner wrote:
Thomas posted an idea a while back about making all of the core Pd objects into a library and having Pd itself just be the very core. I am thinking of doing this for the next Pd-extended, and this got me thinking. It would be great to have a SIMD version of the Pd-core lib as well. This would be easier to deploy and maintain since it could be in /externals/ rather than part of Pd. Then to use it, you would change "-lib pd-core" to "-lib pd-core-simd" or whatever. How does this sound?
I'm all for it.
me too!
T
Am 18.01.2006 um 20:56 schrieb Hans-Christoph Steiner:
Thomas posted an idea a while back about making all of the core Pd objects into a library and having Pd itself just be the very core. I am thinking of doing this for the next Pd-extended, and this got me thinking. It would be great to have a SIMD version of the Pd-core lib as well. This would be easier to deploy and maintain since it could be in /externals/ rather than part of Pd.
what i forgot to mention about the separation of core and internals is that it requires a bit more code restructuring than just linking the object files into two separate binaries. I remember some DSP functions to be associated with internals - this code must be moved over to the core part. But this should be done anyhow.
greetings, Thomas
On Thu, 19 Jan 2006 12:17:58 +0100 Thomas Grill gr@grrrr.org wrote:
what i forgot to mention about the separation of core and internals is that it requires a bit more code restructuring than just linking the object files into two separate binaries. I remember some DSP functions to be associated with internals - this code must be moved over to the core part. But this should be done anyhow.
not to forget the memory alignment ...
t
On Jan 19, 2006, at 6:17 AM, Thomas Grill wrote:
Am 18.01.2006 um 20:56 schrieb Hans-Christoph Steiner:
Thomas posted an idea a while back about making all of the core Pd objects into a library and having Pd itself just be the very core. I am thinking of doing this for the next Pd-extended, and this got me thinking. It would be great to have a SIMD version of the Pd-core lib as well. This would be easier to deploy and maintain since it could be in /externals/ rather than part of Pd.
what i forgot to mention about the separation of core and internals is that it requires a bit more code restructuring than just linking the object files into two separate binaries. I remember some DSP functions to be associated with internals - this code must be moved over to the core part. But this should be done anyhow.
Since this will probably require a lot of reorganization to the Pd code if its to be included in Miller's sources, I think we should probably do it incrementally and leave the above parts to last, since they are the trickiest.
Also, would it be possible and not too much work to make the pd-core lib single-file objects? It would be more flexible if we could have the pd-core objects as part of the namespace, allowing basically any object name to be overridden. This would allow people to make their own custom versions of specific pd-core objects more manageably. Then we could have people maybe writing pd-core objects which run on an external DSP, for example, in addition to the SIMD objects.
.hc
________________________________________________________________________ ____
On Thu, 19 Jan 2006, Thomas Grill wrote:
Am 18.01.2006 um 20:56 schrieb Hans-Christoph Steiner:
Thomas posted an idea a while back about making all of the core Pd objects into a library and having Pd itself just be the very core. I am thinking of doing this for the next Pd-extended, and this got me thinking. It would be great to have a SIMD version of the Pd-core lib as well. This would be easier to deploy and maintain since it could be in /externals/ rather than part of Pd.
what i forgot to mention about the separation of core and internals is that it requires a bit more code restructuring than just linking the object files into two separate binaries. I remember some DSP functions to be associated with internals - this code must be moved over to the core part. But this should be done anyhow.
I expect some object classes to not be really separable from the core, and I don't necessarily mean DSP: think of [inlet], [outlet], [pd]...
However, I expect 80% or 90% of the internal classes to be externalizable. Let's do it for those classes first. Reuse the work of PDa if it makes sense.
BTW, about PDa, it would make sense to make the Pd source C++-compatible, because if we can include C++ code in Pd, it's possible to introduce a fixed-point type that could be typedef'd as t_float, so that for almost all classes there would be no need to explicitly add integer code. (I haven't looked at PDa so I don't know whether it already does that...)
I had made some C++-related fixes but I think that they were part of the same batch of removal of unused variables and those changes have been reverted.
Maybe this ought to be a topic in the pd-dev meeting.
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju | Freelance Digital Arts Engineer, Montréal QC Canada
On Jan 19, 2006, at 4:59 PM, Mathieu Bouchard wrote:
On Thu, 19 Jan 2006, Thomas Grill wrote:
Am 18.01.2006 um 20:56 schrieb Hans-Christoph Steiner:
Thomas posted an idea a while back about making all of the core Pd objects into a library and having Pd itself just be the very core. I am thinking of doing this for the next Pd-extended, and this got me thinking. It would be great to have a SIMD version of the Pd-core lib as well. This would be easier to deploy and maintain since it could be in /externals/ rather than part of Pd.
what i forgot to mention about the separation of core and internals is that it requires a bit more code restructuring than just linking the object files into two separate binaries. I remember some DSP functions to be associated with internals - this code must be moved over to the core part. But this should be done anyhow.
I expect some object classes to not be really separable from the core, and I don't necessarily mean DSP: think of [inlet], [outlet], [pd]...
However, I expect 80% or 90% of the internal classes to be externalizable. Let's do it for those classes first. Reuse the work of PDa if it makes sense.
BTW, about PDa, it would make sense to make the Pd source C++-compatible, because if we can include C++ code in Pd, it's possible to introduce a fixed-point type that could be typedef'd as t_float, so that for almost all classes there would be no need to explicitly add integer code. (I haven't looked at PDa so I don't know whether it already does that...)
I had made some C++-related fixes but I think that they were part of the same batch of removal of unused variables and those changes have been reverted.
Maybe this ought to be a topic in the pd-dev meeting.
Obviously some objects would always remain in the Pd "kernel", like C has void, while, if, else, static, sizeof, typedef, etc.
Perhaps a better route would be to make the pd-core lib. But then make new libs with those objects and use SIMD in them. For example a "math" lib would make a lot of sense, I don't know if there should be "math~" as well, or whether everything should just be in "math", but that's the general idea. "timing" is another idea. "core" could be objects like [trigger], [select], [route], [symbol], [float], [print], etc. "gui" could be another lib.
.hc
________________________________________________________________________ ____
"Looking at things from a more basic level, you can come up with a more direct solution... It may sound small in theory, but it in practice, it can change entire economies." - Amy Smith
On Fri, 20 Jan 2006, Hans-Christoph Steiner wrote:
Perhaps a better route would be to make the pd-core lib. But then make new libs with those objects and use SIMD in them. For example a "math" lib would make a lot of sense, I don't know if there should be "math~" as well, or whether everything should just be in "math", but that's the general idea.
IMHO, [+] and [+~] should be merged in one class. Short of doing that, they should share as much code as possible. Likewise for other float-vs-signal object pairs.
IMHO, the whole code for those classes and more, would be like:
MAKE_ARITH_CLASS("+", x+y) MAKE_ARITH_CLASS("-", x-y) MAKE_ARITH_CLASS("*", x*y) MAKE_ARITH_CLASS("/", x/y)
and that would make both float-classes and signal-classes, supposing that the appropriate macro is defined.
But supposing we use devel's SIMD functions, it would be more like:
MAKE_ARITH_CLASS("+", plus) MAKE_ARITH_CLASS("-", minus) MAKE_ARITH_CLASS("*", times) MAKE_ARITH_CLASS("/", over)
to match the names of the functions.
In practice, two macro invocations have to be used per class, because of limitations of C: one for defining the class's specific functions, and one for registering with Pd inside the _setup.
"timing" is another idea. "core" could be objects like [trigger], [select], [route], [symbol], [float], [print], etc. "gui" could be another lib.
You know, jMax was modularized like that. Maybe you could look at how its libraries were organised =)
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju | Freelance Digital Arts Engineer, Montréal QC Canada
On Thu, 19 Jan 2006, Mathieu Bouchard wrote:
However, I expect 80% or 90% of the internal classes to be externalizable. Let's do it for those classes first. Reuse the work of PDa if it makes sense.
There is probably not that much that you can actually reuse. For a start, I think externalizing the ~ objects makes most sense. These are also the objects that will benefit from SIMD, the message based objects won't run any faster because their performance is bottlenecked by scheduling and other factors that can't take advantage of SIMD.
Instead of the typedef, what I did was to redefine t_sample. Unfortunately the usage of t_sample as the base DSP type is not fully consistent in Pd, which means you have to change several occurences of t_float or float directly to t_sample in the dsp _perform routines.
It would be nice to have them consistent, because we could redefine t_sample to "double" and compile a high-end version of Pd. I admit that I doubt that it would really improve sound quality, but its a good point for selling.
Like, you know, Pd sounds better because it runs with 64 bit numbers.
Guenter
BTW, about PDa, it would make sense to make the Pd source C++-compatible, because if we can include C++ code in Pd, it's possible to introduce a fixed-point type that could be typedef'd as t_float, so that for almost all classes there would be no need to explicitly add integer code. (I haven't looked at PDa so I don't know whether it already does that...)
I had made some C++-related fixes but I think that they were part of the same batch of removal of unused variables and those changes have been reverted.
Maybe this ought to be a topic in the pd-dev meeting.
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju | Freelance Digital Arts Engineer, Montréal QC Canada
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
On Jan 20, 2006, at 4:26 AM, geiger wrote:
On Thu, 19 Jan 2006, Mathieu Bouchard wrote:
However, I expect 80% or 90% of the internal classes to be externalizable. Let's do it for those classes first. Reuse the work of PDa if it makes sense.
There is probably not that much that you can actually reuse. For a start, I think externalizing the ~ objects makes most sense. These are also the objects that will benefit from SIMD, the message based objects won't run any faster because their performance is bottlenecked by scheduling and other factors that can't take advantage of SIMD.
Instead of the typedef, what I did was to redefine t_sample. Unfortunately the usage of t_sample as the base DSP type is not fully consistent in Pd, which means you have to change several occurences of t_float or float directly to t_sample in the dsp _perform routines.
It would be nice to have them consistent, because we could redefine t_sample to "double" and compile a high-end version of Pd. I admit that I doubt that it would really improve sound quality, but its a good point for selling.
Please submit patches to the tracker to do this! It sounds like it makes a lot of sense, and Miller has been accepting most patches submitted these days.
Like, you know, Pd sounds better because it runs with 64 bit numbers.
Actually, 64-bit floats would be nice for things like timestamps. The current floats aren't big enough.
.hc
Guenter
BTW, about PDa, it would make sense to make the Pd source C++-compatible, because if we can include C++ code in Pd, it's possible to introduce a fixed-point type that could be typedef'd as t_float, so that for almost all classes there would be no need to explicitly add integer code. (I haven't looked at PDa so I don't know whether it already does that...)
I had made some C++-related fixes but I think that they were part of the same batch of removal of unused variables and those changes have been reverted.
Maybe this ought to be a topic in the pd-dev meeting.
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju | Freelance Digital Arts Engineer, Montréal QC Canada
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
________________________________________________________________________ ____
There is no way to peace, peace is the way. -A.J. Muste
On Fri, 20 Jan 2006 11:32:26 -0500 Hans-Christoph Steiner hans@eds.org wrote:
Like, you know, Pd sounds better because it runs with 64 bit numbers.
Actually, 64-bit floats would be nice for things like timestamps. The current floats aren't big enough.
some filters would benefit from 64-bit floats because it would reduce quantization effects ...
tim
On Fri, 20 Jan 2006, Hans-Christoph Steiner wrote:
It would be nice to have them consistent, because we could redefine t_sample to "double" and compile a high-end version of Pd. I admit that I doubt that it would really improve sound quality, but its a good point for selling.
Please submit patches to the tracker to do this! It sounds like it makes a lot of sense, and Miller has been accepting most patches submitted these days.
The only problem is that I do not have a patch against the current version of Pd, and there are some cases where I do not know for sure if the right type would be t_sample, t_float or t_floatarg.
Günter
Like, you know, Pd sounds better because it runs with 64 bit numbers.
Actually, 64-bit floats would be nice for things like timestamps. The current floats aren't big enough.
.hc
Guenter
BTW, about PDa, it would make sense to make the Pd source C++-compatible, because if we can include C++ code in Pd, it's possible to introduce a fixed-point type that could be typedef'd as t_float, so that for almost all classes there would be no need to explicitly add integer code. (I haven't looked at PDa so I don't know whether it already does that...)
I had made some C++-related fixes but I think that they were part of the same batch of removal of unused variables and those changes have been reverted.
Maybe this ought to be a topic in the pd-dev meeting.
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju | Freelance Digital Arts Engineer, Montréal QC Canada
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
There is no way to peace, peace is the way. -A.J. Muste
On Jan 21, 2006, at 1:40 PM, geiger wrote:
On Fri, 20 Jan 2006, Hans-Christoph Steiner wrote:
It would be nice to have them consistent, because we could redefine t_sample to "double" and compile a high-end version of Pd. I admit that I doubt that it would really improve sound quality, but its a good point for selling.
Please submit patches to the tracker to do this! It sounds like it makes a lot of sense, and Miller has been accepting most patches submitted these days.
The only problem is that I do not have a patch against the current version of Pd, and there are some cases where I do not know for sure if the right type would be t_sample, t_float or t_floatarg.
It seems to me that the this is kind of a bugfix, I think consistency in code is quite important, so these fixes could probably be submitted incrementally. So if you have a set of fixes that work well now, perhaps it would make sense to submit a patch.
.hc
Günter
Like, you know, Pd sounds better because it runs with 64 bit numbers.
Actually, 64-bit floats would be nice for things like timestamps. The current floats aren't big enough.
.hc
Guenter
BTW, about PDa, it would make sense to make the Pd source C++-compatible, because if we can include C++ code in Pd, it's possible to introduce a fixed-point type that could be typedef'd as t_float, so that for almost all classes there would be no need to explicitly add integer code. (I haven't looked at PDa so I don't know whether it already does that...)
I had made some C++-related fixes but I think that they were part of the same batch of removal of unused variables and those changes have been reverted.
Maybe this ought to be a topic in the pd-dev meeting.
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju | Freelance Digital Arts Engineer, Montréal QC Canada
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
__ ____
There is no way to peace, peace is the way. -A.J. Muste
________________________________________________________________________ ____
Using ReBirth is like trying to play an 808 with a long stick. -David Zicarelli
Hallo!
I expect some object classes to not be really separable from the core, and I don't necessarily mean DSP: think of [inlet], [outlet], [pd]...
However, I expect 80% or 90% of the internal classes to be externalizable. Let's do it for those classes first. Reuse the work of PDa if it makes sense.
just an other thought: if there is a lib for audio, math, ... (as you proposed) it would be really handy, if also the externals are somehow organized in such libraries, and not by name of the authors ... (I know this is not so easy to achieve, would require a reorganization of the cvs, ...)
anyway, I think this would also be much easier for users and would prevent developers to double things, etc. ...
LG Georg
On Jan 20, 2006, at 5:20 AM, Georg Holzmann wrote:
Hallo!
I expect some object classes to not be really separable from the core, and I don't necessarily mean DSP: think of [inlet], [outlet], [pd]... However, I expect 80% or 90% of the internal classes to be externalizable. Let's do it for those classes first. Reuse the work of PDa if it makes sense.
just an other thought: if there is a lib for audio, math, ... (as you proposed) it would be really handy, if also the externals are somehow organized in such libraries, and not by name of the authors ... (I know this is not so easy to achieve, would require a reorganization of the cvs, ...)
anyway, I think this would also be much easier for users and would prevent developers to double things, etc. ...
Absolutely, but that's a bigger project, largely because it involved so many more people. This is definitely the direction we should go in. But instead of just moving existing objects into new containers, I think we need to design each library first, then reuse code when its helpful.
So like libraries in other languages, Pd's libraries should have consistent terms, naming, and interfaces.
.hc
________________________________________________________________________ ____
If you are not part of the solution, you are part of the problem. - Eldridge Cleaver
:)
Some people on the list say that this is not possible, to have a user-friendly (heirarchical) organization of externals. Hans and I think it is possible, there is a crack at it on the PDDP wiki, feel free to contribute. :)
b.
Georg Holzmann wrote:
Hallo!
I expect some object classes to not be really separable from the core, and I don't necessarily mean DSP: think of [inlet], [outlet], [pd]...
However, I expect 80% or 90% of the internal classes to be externalizable. Let's do it for those classes first. Reuse the work of PDa if it makes sense.
just an other thought: if there is a lib for audio, math, ... (as you proposed) it would be really handy, if also the externals are somehow organized in such libraries, and not by name of the authors ... (I know this is not so easy to achieve, would require a reorganization of the cvs, ...)
anyway, I think this would also be much easier for users and would prevent developers to double things, etc. ...
LG Georg
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
On Wed, 18 Jan 2006, Hans-Christoph Steiner wrote:
Then to use it, you would change "-lib pd-core" to "-lib pd-core-simd" or whatever. How does this sound?
Sounds good, this is how I structured PDa, so it would reduce the differences between PDa and standard pd.
Günter
.hc
"The arc of history bends towards justice." - Dr. Martin Luther King, Jr.
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
On Jan 19, 2006, at 7:58 AM, geiger wrote:
On Wed, 18 Jan 2006, Hans-Christoph Steiner wrote:
Then to use it, you would change "-lib pd-core" to "-lib pd-core-simd" or whatever. How does this sound?
Sounds good, this is how I structured PDa, so it would reduce the differences between PDa and standard pd.
Do you have any documentation on this? It would be good to have Pd and PDa as similar as possible. Plus its always good to have an example to start from.
.hc
Günter
.hc
__ ____
"The arc of history bends towards justice." - Dr. Martin Luther King,Jr.
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
________________________________________________________________________ ____
On Thu, 19 Jan 2006, Hans-Christoph Steiner wrote:
Sounds good, this is how I structured PDa, so it would reduce the differences between PDa and standard pd.
Do you have any documentation on this? It would be good to have Pd and PDa as similar as possible. Plus its always good to have an example to start from.
You can download PDa and look at the source. But it is so trivial to change the behaviour (in some cases), that it is not even necessary.
Just delete the xxx_setup() function from the pd source and move all the xxx_* code into a separate file and treat it as an external.
But then, there are some things which make this a bit more complicated in some cases. e.g. as Thomas noted if you want to reuse code in several "internals/externals". This might be important in the SIMD case, but maybe you can factor out standard operations into macros and reuse those. For PDa I had to come up with a workaround for sine tables that are used by some oscillators.
And, I just made the signal processing objects external up to now, so I don't know what problems will arise with others ...
Guenter
.hc
Günter
.hc
__ ____
"The arc of history bends towards justice." - Dr. Martin Luther King,Jr.
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
http://at.or.at/hans/
PD-dev mailing list PD-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
On Thu, 19 Jan 2006, geiger wrote:
there are some things which make this a bit more complicated in some cases. e.g. as Thomas noted if you want to reuse code in several "internals/externals". This might be important in the SIMD case, but maybe you can factor out standard operations into macros and reuse those. For PDa I had to come up with a workaround for sine tables that are used by some oscillators.
I think that the Pd source doesn't use nearly enough macros or other code-reducing tricks.
The reduction of code isn't so much about making things use less RAM: the RAM excuse is quickly evaporating as even the tiniest computers come with plenty of RAM and even the faster kinds of RAM come in ever more copious amounts (big caches).
The reduction of code is programmer-oriented. I'm not talking about length of identifiers here (this is a separate issue). Every line of code should do something interesting by itself. Code should read like a good story and not like a car. Ever tried to read a car? It's boring. The same damn piston copy-pasted 12 times.
The reduction of code is also documentation-oriented. Once the programmer has been contaminated with the wisdom required to make small code or understand small code, then why wouldn't the programmer explain it to his students in higher-level terms instead of chanting 12 times the same piston as if it were a marathon of Hail-Marys ?
This is why Pd needs a taxonomy of object classes. If I don't get that taxonomy in Pd itself nor in its help files, at least I'll have it in its unit tests.
Once and only once. Once and only once. Once and only once. Three strikes and you refactor. for x in [1,2,3] say: Once and only once
http://c2.com/cgi/wiki/?ThreeStrikesAndYouRefactor
BTW I'm not talking about only inheritance of implementations. The most important thing to me is inheritance of expectations, so that if I name 100 classes that obey the rule "Operator2", then you have just learned something common about 100 classes.
Operator2 means right-inlet is cold, left-inlet is hot, there is a "set" method for using left-inlet as cold, there is a "bang" for explicitly activating the main computation. The main computation only produces one message. That's what "Operator2" means in my taxonomy, and it's that much that hasn't to be stated explicitly in each help patch.
Help patches can be abstractions to be used to by other help patches. Just put a [operator2-help] object in your help patch to indicate that the currently documented class obeys the standard operator2 rules.
Who's against it?
_ _ __ ___ _____ ________ _____________ _____________________ ... | Mathieu Bouchard - tél:+1.514.383.3801 - http://artengine.ca/matju | Freelance Digital Arts Engineer, Montréal QC Canada
-
B. Bogart -
geiger -
Georg Holzmann -
Hans-Christoph Steiner -
Mathieu Bouchard -
Thomas Grill -
Tim Blechmann