Hi , Do anybody is working with openframeoworks and libpd? i would like to develop an application that interpret pixels as sounds using libpd addon on openframeworks. I was wondering which would be the best way for sending images(opencvimages ) or pixels arrrays from openframeworks to pd using libpd and receiving it in pd for interpreting it as sound in real time. Do anybody have tried soemthing like this? any idea?
R.
I think with the libpd API, you can write to Pd arrays. That's
probably you're best bet.
.hc
On Aug 20, 2011, at 6:11 PM, ronni montoya wrote:
Hi , Do anybody is working with openframeoworks and libpd? i would like to develop an application that interpret pixels as sounds using libpd addon on openframeworks. I was wondering which would be the best way for sending images(opencvimages ) or pixels arrrays from openframeworks to pd using libpd and receiving it in pd for interpreting it as sound in real time. Do anybody have tried soemthing like this? any idea?
R.
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Access to computers should be unlimited and total. - the hacker ethic
On Sun, 21 Aug 2011, Hans-Christoph Steiner wrote:
I think with the libpd API, you can write to Pd arrays. That's probably you're best bet.
You must be meaning the pd API (m_pd.h).
There's nothing libpd-specific (z_libpd.h) about arrays.
It's a good thing, because IMHO, many of the functions whose name start with the letters «libpd» are pointless, as they can already be done using typedmess() and such.
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
On Sat, 20 Aug 2011, ronni montoya wrote:
Hi , Do anybody is working with openframeoworks and libpd? i would like to develop an application that interpret pixels as sounds using libpd addon on openframeworks. I was wondering which would be the best way for sending images(opencvimages ) or pixels arrrays from openframeworks to pd using libpd and receiving it in pd for interpreting it as sound in real time. Do anybody have tried soemthing like this? any idea?
You can make yourself tilde externals for pd, that you embed in your libpd-based app... e.g. one or two outlets, no inlets.
You just call the setup-functions of the externals just after initialising libpd... the externals don't need to be separate files (dll, so, dylib) : they can be part of your main executable instead, which is easier.
I already do that with non-tilde externals. (Haven't had a reason to make tilde externals in that context yet).
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
That's a good question.
Hrm, well you can easily read and write to arrays which are naturally 1 dimensional. I would first try flattening the image by writing the pixel buffer into a float array. Then send the width and height as well for reading it back in pd. There currently isn't a way to send an area of memory and I could imagine sending a giant list would be alot slower then using an array.
See the example in https://github.com/danomatika/ofxPd for how to read/write to pd arrays.
Otherwise you could write and external as Mathieu suggests ...
On Aug 27, 2011, at 11:21 PM, Mathieu Bouchard wrote:
On Sat, 20 Aug 2011, ronni montoya wrote:
Hi , Do anybody is working with openframeoworks and libpd? i would like to develop an application that interpret pixels as sounds using libpd addon on openframeworks. I was wondering which would be the best way for sending images(opencvimages ) or pixels arrrays from openframeworks to pd using libpd and receiving it in pd for interpreting it as sound in real time. Do anybody have tried soemthing like this? any idea?
You can make yourself tilde externals for pd, that you embed in your libpd-based app... e.g. one or two outlets, no inlets.
You just call the setup-functions of the externals just after initialising libpd... the externals don't need to be separate files (dll, so, dylib) : they can be part of your main executable instead, which is easier.
I already do that with non-tilde externals. (Haven't had a reason to make tilde externals in that context yet).
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
Dan Wilcox danomatika.com robotcowboy.com
On Sun, 28 Aug 2011, Dan Wilcox wrote:
and I could imagine sending a giant list would be alot slower then using an array.
A list is passed by pointer. The first thing that can be slower, is having to read the atomtype because list elements can be things other than floats. The second thing that can be slower, is if ever you need to copy the list or part thereof. But until the method returns, the argc and argv parametres can be used directly.
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
I was talking about sending a large list through libpd which, I assume, is doing some sort of copying of floats as it translates from the libpd api to the internal pd api. At least that's what my ofxPD wrapper does, passes each float by value to libpd. Someone correct me if I'm wrong.
Besides, isn't there some sort of limit on the length of lists or does libpd handle this for you?
On Aug 28, 2011, at 1:33 PM, Mathieu Bouchard wrote:
On Sun, 28 Aug 2011, Dan Wilcox wrote:
and I could imagine sending a giant list would be alot slower then using an array.
A list is passed by pointer. The first thing that can be slower, is having to read the atomtype because list elements can be things other than floats. The second thing that can be slower, is if ever you need to copy the list or part thereof. But until the method returns, the argc and argv parametres can be used directly.
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
Dan Wilcox danomatika.com robotcowboy.com
On Mon, 29 Aug 2011, Dan Wilcox wrote:
I was talking about sending a large list through libpd which, I assume, is doing some sort of copying of floats as it translates from the libpd api to the internal pd api.
Oh, yeah... if you use z_libpd.h, it copies the floats, whereas with m_pd.h, you have the option to have your data as atoms in the first place, so that they don't have to be copied.
Besides, isn't there some sort of limit on the length of lists or does libpd handle this for you?
With z_libpd.h, you have an artificial limit of 32 atoms per message, whereas with m_pd.h, there's no formal limit ; in practice the OS will limit you to something like 1000000 atoms on the stack at once, for example. On 32-bit Linux, the limit can be as high as 8000000 atoms (64 megs) but I don't remember what the default limits are.
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
----- Original Message -----
From: Mathieu Bouchard matju@artengine.ca To: Dan Wilcox danomatika@gmail.com Cc: PD List pd-list@iem.at Sent: Monday, August 29, 2011 12:54 AM Subject: Re: [PD] sending image from of / libpd
On Mon, 29 Aug 2011, Dan Wilcox wrote:
I was talking about sending a large list through libpd which, I assume, is
doing some sort of copying of floats as it translates from the libpd api to the internal pd api.
Oh, yeah... if you use z_libpd.h, it copies the floats, whereas with m_pd.h, you have the option to have your data as atoms in the first place, so that they don't have to be copied.
Besides, isn't there some sort of limit on the length of lists or does
libpd handle this for you?
With z_libpd.h, you have an artificial limit of 32 atoms per message
I must be misunderstanding what you've written, because it would break trivial patches:
[osc~ 440] | | [bang( |/ [print~]
whereas with m_pd.h, there's no formal limit ; in practice the OS will limit you to something like 1000000 atoms on the stack at once, for example. On 32-bit Linux, the limit can be as high as 8000000 atoms (64 megs) but I don't remember what the default limits are.
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
On Sun, 28 Aug 2011, Jonathan Wilkes wrote:
With z_libpd.h, you have an artificial limit of 32 atoms per message
I must be misunderstanding what you've written, because it would break trivial patches:
[osc~ 440] | | [bang( |/ [print~]
Where's the 32 atoms in a message in your example ?
And then, they have to be sent using libpd_add_ functions, because with typedmess or outlet_anything, you have no limit other than the OS' maximum stack size.
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
----- Original Message -----
From: Mathieu Bouchard matju@artengine.ca To: Jonathan Wilkes jancsika@yahoo.com Cc: Dan Wilcox danomatika@gmail.com; PD List pd-list@iem.at Sent: Monday, August 29, 2011 1:19 AM Subject: Re: [PD] sending image from of / libpd
On Sun, 28 Aug 2011, Jonathan Wilkes wrote:
With z_libpd.h, you have an artificial limit of 32 atoms per message
I must be misunderstanding what you've written, because it would break trivial patches:
[osc~ 440] | | [bang( |/ [print~]
Where's the 32 atoms in a message in your example ?
Bad example.
And then, they have to be sent using libpd_add_ functions, because with typedmess or outlet_anything, you have no limit other than the OS' maximum stack size.
Ah, never mind then.
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
No, I'm talking about sending a list or typed message with libpd as in:
[list 1 2 3 4 ... 32 < | [s toC++]
The print messaging isn't limited as far as I know.
I think it's reasonable to limit the message size, but 32 may be too small for some. It would make sense for libpd to have a call that would allow users to set the max message size, akin to how you can set the max packet size on sockets etc
32 is fine for most people, but there are uses for more ...
Also, I imagine you don't have this limitation using the new *t_atom sending func. Is this true Peter?
On Aug 29, 2011, at 1:15 AM, Jonathan Wilkes wrote:
----- Original Message -----
From: Mathieu Bouchard matju@artengine.ca To: Dan Wilcox danomatika@gmail.com Cc: PD List pd-list@iem.at Sent: Monday, August 29, 2011 12:54 AM Subject: Re: [PD] sending image from of / libpd
On Mon, 29 Aug 2011, Dan Wilcox wrote:
I was talking about sending a large list through libpd which, I assume, is
doing some sort of copying of floats as it translates from the libpd api to the internal pd api.
Oh, yeah... if you use z_libpd.h, it copies the floats, whereas with m_pd.h, you have the option to have your data as atoms in the first place, so that they don't have to be copied.
Besides, isn't there some sort of limit on the length of lists or does
libpd handle this for you?
With z_libpd.h, you have an artificial limit of 32 atoms per message
I must be misunderstanding what you've written, because it would break trivial patches:
[osc~ 440] | | [bang( |/ [print~]
whereas with m_pd.h, there's no formal limit ; in practice the OS will limit you to something like 1000000 atoms on the stack at once, for example. On 32-bit Linux, the limit can be as high as 8000000 atoms (64 megs) but I don't remember what the default limits are.
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Dan Wilcox danomatika.com robotcowboy.com
On Mon, Aug 29, 2011 at 1:23 AM, Dan Wilcox danomatika@gmail.com wrote:
No, I'm talking about sending a list or typed message with libpd as in:
[list 1 2 3 4 ... 32 < | [s toC++]
The print messaging isn't limited as far as I know.
That's right. The only part that imposes a limit on the message size is the simple message assembly mechanism for sending lists to Pd. Lists that are created elsewhere have no such limitation.
I think it's reasonable to limit the message size, but 32 may be too small
for some. It would make sense for libpd to have a call that would allow users to set the max message size, akin to how you can set the max packet size on sockets etc
Hmm. I'm reluctant to add a call for changing the array size because this entire message assembly API is really just there for the purpose of simplifying the creation of language bindings. Any additional complexity would make it even harder to justify. So far I've felt that it was justified because it seemed good enough and I believe its simplicity has helped the adoption of libpd. If it turns out to be too limiting (and this thread suggests that that may be the case), then I'll have to bite the bullet, deprecate this approach, and write the manual type conversions for Java and other languages that I'd been trying to avoid. The good news is that the latest version of libpd already comes with a new pair of message passing functions that are not limited in this way.
The current limit was chosen in the following highly scientific fashion: I tried to think of the longest list I might want to send as a message, and the biggest thing I could realistically think of was an OpenGL transformation, i.e. 4x4 = 16 numbers. Then I doubled that count and hoped that that would be enough for everybody.
Maybe we can have a poll and come up with a better estimate. Here are a few questions:
- What sort of use case for long list messages to you have in mind?
- Exactly how long would you need the list to be?
- Is this really a use case for list messages or would it make more sense
to write to an array instead?
If we converge to a reasonable number, that'll be the new limit. Otherwise, the entire approach may have to go.
Also, I imagine you don't have this limitation using the new *t_atom sending func. Is this true Peter?
That's right, there's no intrinsic limitation in the functions. With the new approach, you're responsible for creating the t_atom arrays that you're sending; if you can create it, then the new functions can send it. Cheers, Peter
Whoah whoah hold on. I'm not suggesting to dump the message sending API. I'm only asking for the ability to set the max message size. In most cases 32 is plenty. The *t_atom send func is sufficient for more advanced users. I like the message API and will use it as the default anyway.
enohp ym morf tnes
Dan Wilcox danomatika.com robotcowboy.com
On Aug 29, 2011, at 2:51 PM, Peter Brinkmann peter.brinkmann@googlemail.com wrote:
On Mon, Aug 29, 2011 at 1:23 AM, Dan Wilcox danomatika@gmail.com wrote: No, I'm talking about sending a list or typed message with libpd as in:
[list 1 2 3 4 ... 32 < | [s toC++]
The print messaging isn't limited as far as I know.
That's right. The only part that imposes a limit on the message size is the simple message assembly mechanism for sending lists to Pd. Lists that are created elsewhere have no such limitation.
I think it's reasonable to limit the message size, but 32 may be too small for some. It would make sense for libpd to have a call that would allow users to set the max message size, akin to how you can set the max packet size on sockets etc
Hmm. I'm reluctant to add a call for changing the array size because this entire message assembly API is really just there for the purpose of simplifying the creation of language bindings. Any additional complexity would make it even harder to justify. So far I've felt that it was justified because it seemed good enough and I believe its simplicity has helped the adoption of libpd. If it turns out to be too limiting (and this thread suggests that that may be the case), then I'll have to bite the bullet, deprecate this approach, and write the manual type conversions for Java and other languages that I'd been trying to avoid. The good news is that the latest version of libpd already comes with a new pair of message passing functions that are not limited in this way.
The current limit was chosen in the following highly scientific fashion: I tried to think of the longest list I might want to send as a message, and the biggest thing I could realistically think of was an OpenGL transformation, i.e. 4x4 = 16 numbers. Then I doubled that count and hoped that that would be enough for everybody.
Maybe we can have a poll and come up with a better estimate. Here are a few questions:
- What sort of use case for long list messages to you have in mind?
- Exactly how long would you need the list to be?
- Is this really a use case for list messages or would it make more sense to write to an array instead?
If we converge to a reasonable number, that'll be the new limit. Otherwise, the entire approach may have to go.
Also, I imagine you don't have this limitation using the new *t_atom sending func. Is this true Peter?
That's right, there's no intrinsic limitation in the functions. With the new approach, you're responsible for creating the t_atom arrays that you're sending; if you can create it, then the new functions can send it. Cheers, Peter
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-08-29 20:51, Peter Brinkmann wrote:
Maybe we can have a poll and come up with a better estimate. Here are a few questions:
- What sort of use case for long list messages to you have in mind?
- Exactly how long would you need the list to be?
- Is this really a use case for list messages or would it make more sense
to write to an array instead?
If we converge to a reasonable number, that'll be the new limit. Otherwise, the entire approach may have to go.
personally i do think that any fixed limit is going to impose troubles.
afaict, the main idea here is to provide an easy to use API, combined with some performance.
somebody _will_ reach your hard limit, and i think that telling them to either switch from the "foolproof API" to "export mode" or to redesign their patches just because they called "add_element()" once too often will not do.
why not simply resize the internal array as needed, starting with 32 elements and doubling whenever the limit is reached?
fgasdr IOhannes
i solved this creating a fuction called send inside AppCore:
void AppCore::send(unsigned char *pixels) {
for(int i = 0; i < array1.size(); i++)
array1[i] = pixels[i]* 1;
pd.writeArray("array1", array1);
}
then im calling the function from testApp.cpp and sending the pixels from a opecv gray image.
void testApp::draw() { core.send(grayImage.getPixels());
}
What do you guys think, maybe theres a more effective way?
cheers
R.
2011/8/30 IOhannes m zmoelnig zmoelnig@iem.at:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-08-29 20:51, Peter Brinkmann wrote:
Maybe we can have a poll and come up with a better estimate. Here are a few questions:
- What sort of use case for long list messages to you have in mind?
- Exactly how long would you need the list to be?
- Is this really a use case for list messages or would it make more sense
to write to an array instead?
If we converge to a reasonable number, that'll be the new limit. Otherwise, the entire approach may have to go.
personally i do think that any fixed limit is going to impose troubles.
afaict, the main idea here is to provide an easy to use API, combined with some performance.
somebody _will_ reach your hard limit, and i think that telling them to either switch from the "foolproof API" to "export mode" or to redesign their patches just because they called "add_element()" once too often will not do.
why not simply resize the internal array as needed, starting with 32 elements and doubling whenever the limit is reached?
fgasdr IOhannes -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk5ckYcACgkQkX2Xpv6ydvQ85gCfWJcNIzEGnhz9hdnz/XBuRYuI TMEAnjfytStl5sFDqkJ9HppZ9XQylgni =HvJ0 -----END PGP SIGNATURE-----
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
For gray values this should work.
On Aug 30, 2011, at 6:56 AM, ronni montoya wrote:
i solved this creating a fuction called send inside AppCore:
void AppCore::send(unsigned char *pixels) {
for(int i = 0; i < array1.size(); i++) array1[i] = pixels[i]* 1;
pd.writeArray("array1", array1); }
Here I would send a resize message to the array if the array length is different from you image size. Also, I'm assuming you set the correct size for the array1 vector to be the size of the image pixel buffer aka width*height ...
void AppCore::send(unsigned char *pixels) {
// check array length
if(array1.size() != pd.getArrayLen("array1") {
//
// send resize message to array1
//
// aka [ ; array1 resize # <
//
pd. << StartMsg("array1") << "resize" << pixelLen << Finish();
}
for(int i = 0; i < array1.size(); i++)
array1[i] = pixels[i];
pd.writeArray("array1", array1);
}
then im calling the function from testApp.cpp and sending the pixels from a opecv gray image.
void testApp::draw() { core.send(grayImage.getPixels());
}
I'd send the image to pd in the core.update() function. The whole point of the AppCore wrapper classes is to be able to write the same core app code for both desktop and iOS, since iOS requires a Obj-C file for the main Application delegate aka example/src/ios/testApp.m.
What do you guys think, maybe theres a more effective way?
Dunno. Try it out and let us know. I'm pretty sure it may be slow to send a largish image > 640x480, so I'd start out at 120x240 or so. I've streamed realtime color video at that size using binary blobs in OSC without a problem, so I can't imagine small images being too slow ... then again I haven't tried it with libpd.
cheers
R.
2011/8/30 IOhannes m zmoelnig zmoelnig@iem.at:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-08-29 20:51, Peter Brinkmann wrote:
Maybe we can have a poll and come up with a better estimate. Here are a few questions:
- What sort of use case for long list messages to you have in mind?
- Exactly how long would you need the list to be?
- Is this really a use case for list messages or would it make more sense
to write to an array instead?
If we converge to a reasonable number, that'll be the new limit. Otherwise, the entire approach may have to go.
personally i do think that any fixed limit is going to impose troubles.
afaict, the main idea here is to provide an easy to use API, combined with some performance.
somebody _will_ reach your hard limit, and i think that telling them to either switch from the "foolproof API" to "export mode" or to redesign their patches just because they called "add_element()" once too often will not do.
why not simply resize the internal array as needed, starting with 32 elements and doubling whenever the limit is reached?
fgasdr IOhannes -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk5ckYcACgkQkX2Xpv6ydvQ85gCfWJcNIzEGnhz9hdnz/XBuRYuI TMEAnjfytStl5sFDqkJ9HppZ9XQylgni =HvJ0 -----END PGP SIGNATURE-----
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Dan Wilcox danomatika.com robotcowboy.com
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-08-30 14:45, Dan Wilcox wrote:
Dunno. Try it out and let us know. I'm pretty sure it may be slow to send a largish image > 640x480, so I'd start out at 120x240 or so. I've streamed realtime color video at that size using binary blobs in OSC without a problem, so I can't imagine small images being too slow ... then again I haven't tried it with libpd.
i assume that is on localhost... if doing that on a real network with OSC-over-UDP (which i assume as well, as it is the default), you will soon get into problems: even with jumbo frames turned on, the maximum frame size is 9000bytes.
fgamsdr IOhannes
Hah, oh IOhannes ...
Go to Ars Electronica and you'll see a little pillar next to the Donau that's running from 9pm to 10pm every night. The live display is actually rendered on the server deep inside the building and sent over OSC to the pillar. It's been running since last August ...
Most osc implementations allow you to increase the max packet size.
On Aug 30, 2011, at 9:10 AM, IOhannes m zmoelnig wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-08-30 14:45, Dan Wilcox wrote:
Dunno. Try it out and let us know. I'm pretty sure it may be slow to send a largish image > 640x480, so I'd start out at 120x240 or so. I've streamed realtime color video at that size using binary blobs in OSC without a problem, so I can't imagine small images being too slow ... then again I haven't tried it with libpd.
i assume that is on localhost... if doing that on a real network with OSC-over-UDP (which i assume as well, as it is the default), you will soon get into problems: even with jumbo frames turned on, the maximum frame size is 9000bytes.
fgamsdr IOhannes -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk5c4TYACgkQkX2Xpv6ydvRHtwCg7pP0UpqV9Yv63okqo4O1Lcc/ xlUAoNxKdheE8lL+HwOQ3dCEWyeqx+kY =U42r -----END PGP SIGNATURE-----
Dan Wilcox danomatika.com robotcowboy.com
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-08-30 17:15, Dan Wilcox wrote:
Hah, oh IOhannes ...
Go to Ars Electronica and you'll see a little pillar next to the Donau that's running from 9pm to 10pm every night. The live display is actually rendered on the server deep inside the building and sent over OSC to the pillar. It's been running since last August ...
Most osc implementations allow you to increase the max packet size.
i am not aware of an OSC implementation that can increase the packet size of the kernel and of all the external hardware (routers) involved.
do you remember the actually used packet size? was that UDP?
fgmasdr IOhannes
Doh, ok sorry for being an ass ... on second thought I believe it's more like 80x60 RGB. I was reminded that the facade framebuffer is much lower res then 240x120 ... The OSC datagram size is set to 20000 in Processing (oscP5), so 20 kB. OSC is designed to handle packet sizes for you, so I imagine it handles breaking things down between individual packets.
On Aug 30, 2011, at 9:10 AM, IOhannes m zmoelnig wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-08-30 14:45, Dan Wilcox wrote:
Dunno. Try it out and let us know. I'm pretty sure it may be slow to send a largish image > 640x480, so I'd start out at 120x240 or so. I've streamed realtime color video at that size using binary blobs in OSC without a problem, so I can't imagine small images being too slow ... then again I haven't tried it with libpd.
i assume that is on localhost... if doing that on a real network with OSC-over-UDP (which i assume as well, as it is the default), you will soon get into problems: even with jumbo frames turned on, the maximum frame size is 9000bytes.
fgamsdr IOhannes -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk5c4TYACgkQkX2Xpv6ydvRHtwCg7pP0UpqV9Yv63okqo4O1Lcc/ xlUAoNxKdheE8lL+HwOQ3dCEWyeqx+kY =U42r -----END PGP SIGNATURE-----
Dan Wilcox danomatika.com robotcowboy.com
On Tue, 30 Aug 2011, IOhannes m zmoelnig wrote:
why not simply resize the internal array as needed, starting with 32 elements and doubling whenever the limit is reached?
binbuf could be used for that, if the libc's realloc does in effect the same thing. i think that this is the case with modern glibc (Linux/CygWin) as well as Perl's allocator (when I looked at it 10 years ago), but not necessarily other libc implementations (FreeBSD/OSX, Google, Microsoft) that I didn't try it with, so if anyone could check, that would be interesting.
To test that, add millions of atoms to a binbuf and see how much time it takes. A «bad» implementation will roughly quadruple the time each time you double the number individual additions, whereas a «good» one will just double the time.
But in «bad» implementations, the assumption was that the user of realloc was responsible to manage extra preallocations such that the buffer wouldn't switch places often... it's not really bad, just inconvenient.
I think that those facts are not very well known... perhaps poorly advertised. People porting Pd would have an interest in benchmarking some parts of pd in case they assume too much about realloc on non-glibc platforms.
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
On Tue, Aug 30, 2011 at 12:51 PM, Mathieu Bouchard matju@artengine.cawrote:
On Tue, 30 Aug 2011, IOhannes m zmoelnig wrote:
why not simply resize the internal array as needed, starting with 32
elements and doubling whenever the limit is reached?
binbuf could be used for that, if the libc's realloc does in effect the same thing. i think that this is the case with modern glibc (Linux/CygWin) as well as Perl's allocator (when I looked at it 10 years ago), but not necessarily other libc implementations (FreeBSD/OSX, Google, Microsoft) that I didn't try it with, so if anyone could check, that would be interesting.
To test that, add millions of atoms to a binbuf and see how much time it takes. A «bad» implementation will roughly quadruple the time each time you double the number individual additions, whereas a «good» one will just double the time.
For the time being, I have something much simpler in mind: Just take the current call "int libpd_start_message(void)", which returns the current limit, and replace it with "int libpd_start_message(int length)", which takes a parameter indicating the length of the message and returns a nonzero error code if the length is too big.
I like this idea because it's almost as simple as the original approach, and the API agnostic with respect to the policy for handling long messages. The tentative implementation that I have right now still has the hard-coded limit of 32 elements, and libpd_start_message simply returns an error code when asked for a message with more than 32 elements. The next revision will dynamically resize the array if the requested length exceeds the current size, and it will only return an error code if the allocation of a larger array fails. Either way, there's no need to reallocate because the array contains nothing of interest when libpd_start_message is called, and so we can simply throw away the old array and allocate a new one.
The only disadvantage I see is that this doesn't sit well with the streaming-style C++ wrapper that Dan created for ofxPd, but I can think of workarounds for that. Cheers, Peter
On Tue, 30 Aug 2011, Peter Brinkmann wrote:
For the time being, I have something much simpler in mind: Just take the current call "int libpd_start_message(void)", which returns the current limit, and replace it with "int libpd_start_message(int length)", which takes a parameter indicating the length of the message and returns a nonzero error code if the length is too big.
But this means that new libpd-using apps won't compile with old versions of libpd AND vice-versa.
... unless they stick to <m_pd.h>, of course.
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
On Tue, Aug 30, 2011 at 3:44 PM, Mathieu Bouchard matju@artengine.cawrote:
On Tue, 30 Aug 2011, Peter Brinkmann wrote:
For the time being, I have something much simpler in mind: Just take the
current call "int libpd_start_message(void)", which returns the current limit, and replace it with "int libpd_start_message(int length)", which takes a parameter indicating the length of the message and returns a nonzero error code if the length is too big.
But this means that new libpd-using apps won't compile with old versions of libpd AND vice-versa.
Well, the vast majority of users won't notice any difference at all because they're using the Android or iOS branch, which I'm updating as I go along. The only people who are affected by this are those who are using the C library directly, and I hope that they'll either be willing to update their code (which should be no more than a two-line change in most cases) or just stick to the current version, which will remain available via git.
In any case, I think everybody understands that this is still a young library that needs to adapt as we gain a better understanding of how people are using it, and the cost of making a small incompatible change is a lot lower than choosing a suboptimal solution for compatibility with an earlier version. This period of youthful innocence is coming to an end, though; the API has been quite stable for quite a while now, and I believe that it'll soon be time to declare it finished. I want to take a critical look at every piece before we officially lock the API, and I won't be afraid to cut things that may turn out to be a burden in the long run. (That's why I floated the idea of getting rid of the simple message assembly mechanism, but it looks like that's here to stay.) Cheers, Peter
I think it's much simpler to just add a call to get/set the message limit, say:
int libpd_max_message_length(); void libpd_set_max_message_length(int length);
This doesn't break any current code.
Having to set a custom limit each time is far more tedious then just setting it at startup.
On Aug 30, 2011, at 5:47 PM, Peter Brinkmann wrote:
On Tue, Aug 30, 2011 at 3:44 PM, Mathieu Bouchard matju@artengine.ca wrote: On Tue, 30 Aug 2011, Peter Brinkmann wrote:
For the time being, I have something much simpler in mind: Just take the current call "int libpd_start_message(void)", which returns the current limit, and replace it with "int libpd_start_message(int length)", which takes a parameter indicating the length of the message and returns a nonzero error code if the length is too big.
But this means that new libpd-using apps won't compile with old versions of libpd AND vice-versa.
Well, the vast majority of users won't notice any difference at all because they're using the Android or iOS branch, which I'm updating as I go along. The only people who are affected by this are those who are using the C library directly, and I hope that they'll either be willing to update their code (which should be no more than a two-line change in most cases) or just stick to the current version, which will remain available via git.
In any case, I think everybody understands that this is still a young library that needs to adapt as we gain a better understanding of how people are using it, and the cost of making a small incompatible change is a lot lower than choosing a suboptimal solution for compatibility with an earlier version. This period of youthful innocence is coming to an end, though; the API has been quite stable for quite a while now, and I believe that it'll soon be time to declare it finished. I want to take a critical look at every piece before we officially lock the API, and I won't be afraid to cut things that may turn out to be a burden in the long run. (That's why I floated the idea of getting rid of the simple message assembly mechanism, but it looks like that's here to stay.) Cheers, Peter
Dan Wilcox danomatika.com robotcowboy.com
On Wed, Aug 31, 2011 at 6:25 AM, Dan Wilcox danomatika@gmail.com wrote:
I think it's much simpler to just add a call to get/set the message limit, say:
int libpd_max_message_length(); void libpd_set_max_message_length(int length);
This doesn't break any current code.
Having to set a custom limit each time is far more tedious then just setting it at startup.
Actually, breakage of current code is a feature as far as I am concerned because it makes people aware of the change, and it should be harmless because it's easy to fix. The language bindings for Java and Objective-C actually became simpler when I updated them for the new version.
I don't think the new signature of libpd_start_message is tedious, really. Essentially, I see two use cases: Either you know an a-priori limit on your message length, in which case there's the tiny extra effort of passing in the limit every time you start a message, or you don't have an a-priori limit, in which case you need to check the length before assembling a message anyway.
Another aspect is API design. One feature of a good API is that it's difficult to use incorrectly. With a separate call for setting the message limit, people will forget that the limit is a consideration. With the current solution, people will briefly contemplate the length of each message they start, which is a good thing. Cheers, Peter
On Aug 30, 2011, at 5:47 PM, Peter Brinkmann wrote:
On Tue, Aug 30, 2011 at 3:44 PM, Mathieu Bouchard matju@artengine.cawrote:
On Tue, 30 Aug 2011, Peter Brinkmann wrote:
For the time being, I have something much simpler in mind: Just take the
current call "int libpd_start_message(void)", which returns the current limit, and replace it with "int libpd_start_message(int length)", which takes a parameter indicating the length of the message and returns a nonzero error code if the length is too big.
But this means that new libpd-using apps won't compile with old versions of libpd AND vice-versa.
Well, the vast majority of users won't notice any difference at all because they're using the Android or iOS branch, which I'm updating as I go along. The only people who are affected by this are those who are using the C library directly, and I hope that they'll either be willing to update their code (which should be no more than a two-line change in most cases) or just stick to the current version, which will remain available via git.
In any case, I think everybody understands that this is still a young library that needs to adapt as we gain a better understanding of how people are using it, and the cost of making a small incompatible change is a lot lower than choosing a suboptimal solution for compatibility with an earlier version. This period of youthful innocence is coming to an end, though; the API has been quite stable for quite a while now, and I believe that it'll soon be time to declare it finished. I want to take a critical look at every piece before we officially lock the API, and I won't be afraid to cut things that may turn out to be a burden in the long run. (That's why I floated the idea of getting rid of the simple message assembly mechanism, but it looks like that's here to stay.) Cheers, Peter
Dan Wilcox danomatika.com robotcowboy.com
On Aug 31, 2011, at 8:45 AM, Peter Brinkmann wrote:
On Wed, Aug 31, 2011 at 6:25 AM, Dan Wilcox danomatika@gmail.com wrote: I think it's much simpler to just add a call to get/set the message limit, say:
int libpd_max_message_length(); void libpd_set_max_message_length(int length);
This doesn't break any current code.
Having to set a custom limit each time is far more tedious then just setting it at startup.
Actually, breakage of current code is a feature as far as I am concerned because it makes people aware of the change, and it should be harmless because it's easy to fix. The language bindings for Java and Objective-C actually became simpler when I updated them for the new version.
I don't think the new signature of libpd_start_message is tedious, really. Essentially, I see two use cases: Either you know an a-priori limit on your message length, in which case there's the tiny extra effort of passing in the limit every time you start a message, or you don't have an a-priori limit, in which case you need to check the length before assembling a message anyway.
Another aspect is API design. One feature of a good API is that it's difficult to use incorrectly. With a separate call for setting the message limit, people will forget that the limit is a consideration. With the current solution, people will briefly contemplate the length of each message they start, which is a good thing.
... but you can simply return an error or print a message complaining when the message is too long. My whole point is that most people won't bother changing the limit and those that do will just pick a larger size with plenty of space anyway. It's too much work to bother setting it EACH and EVERY time. It's far LESS elegant, and dare I say intuitive. It seems like an unnecessary step. [eople that have problems will only run into this once, increase the max size, and then be fine. Why force them to compute a size manually each time when they could just be happily adding objects ... ?
In case, my wrapper will include max message size get/set functions and complain to cout when something is out of bounds.
Cheers, Peter
On Aug 30, 2011, at 5:47 PM, Peter Brinkmann wrote:
On Tue, Aug 30, 2011 at 3:44 PM, Mathieu Bouchard matju@artengine.ca wrote: On Tue, 30 Aug 2011, Peter Brinkmann wrote:
For the time being, I have something much simpler in mind: Just take the current call "int libpd_start_message(void)", which returns the current limit, and replace it with "int libpd_start_message(int length)", which takes a parameter indicating the length of the message and returns a nonzero error code if the length is too big.
But this means that new libpd-using apps won't compile with old versions of libpd AND vice-versa.
Well, the vast majority of users won't notice any difference at all because they're using the Android or iOS branch, which I'm updating as I go along. The only people who are affected by this are those who are using the C library directly, and I hope that they'll either be willing to update their code (which should be no more than a two-line change in most cases) or just stick to the current version, which will remain available via git.
In any case, I think everybody understands that this is still a young library that needs to adapt as we gain a better understanding of how people are using it, and the cost of making a small incompatible change is a lot lower than choosing a suboptimal solution for compatibility with an earlier version. This period of youthful innocence is coming to an end, though; the API has been quite stable for quite a while now, and I believe that it'll soon be time to declare it finished. I want to take a critical look at every piece before we officially lock the API, and I won't be afraid to cut things that may turn out to be a burden in the long run. (That's why I floated the idea of getting rid of the simple message assembly mechanism, but it looks like that's here to stay.) Cheers, Peter
Dan Wilcox danomatika.com robotcowboy.com
Dan Wilcox danomatika.com robotcowboy.com
On Wed, Aug 31, 2011 at 9:01 AM, Dan Wilcox danomatika@gmail.com wrote:
On Aug 31, 2011, at 8:45 AM, Peter Brinkmann wrote:
On Wed, Aug 31, 2011 at 6:25 AM, Dan Wilcox danomatika@gmail.com wrote:
I think it's much simpler to just add a call to get/set the message limit, say:
int libpd_max_message_length(); void libpd_set_max_message_length(int length);
This doesn't break any current code.
Having to set a custom limit each time is far more tedious then just setting it at startup.
Actually, breakage of current code is a feature as far as I am concerned because it makes people aware of the change, and it should be harmless because it's easy to fix. The language bindings for Java and Objective-C actually became simpler when I updated them for the new version.
I don't think the new signature of libpd_start_message is tedious, really. Essentially, I see two use cases: Either you know an a-priori limit on your message length, in which case there's the tiny extra effort of passing in the limit every time you start a message, or you don't have an a-priori limit, in which case you need to check the length before assembling a message anyway.
Another aspect is API design. One feature of a good API is that it's difficult to use incorrectly. With a separate call for setting the message limit, people will forget that the limit is a consideration. With the current solution, people will briefly contemplate the length of each message they start, which is a good thing.
... but you can simply return an error or print a message complaining when the message is too long. My whole point is that most people won't bother changing the limit and those that do will just pick a larger size with plenty of space anyway. It's too much work to bother setting it EACH and EVERY time. It's far LESS elegant, and dare I say intuitive. It seems like an unnecessary step. [eople that have problems will only run into this once, increase the max size, and then be fine. Why force them to compute a size manually each time when they could just be happily adding objects ... ?
Actually, I don't mean to make people compute the size every time The length parameter in libpd_start_message merely makes sure that you'll have enough space for that many elements; the number of elements in the actual message may be smaller. If you know that your messages will never exceed a certain length, then you can simply use that maximum length every time. In that situation, I would just wrap the call to libpd_start_message in a macro or a convenience function to get exactly the effect that you want.
In case, my wrapper will include max message size get/set functions and complain to cout when something is out of bounds.
And with the streaming API that you've built in C++, that's the right thing to do:) Cheers, Peter
On Wed, Aug 31, 2011 at 11:27 AM, Peter Brinkmann < peter.brinkmann@googlemail.com> wrote:
On Wed, Aug 31, 2011 at 9:01 AM, Dan Wilcox danomatika@gmail.com wrote:
On Aug 31, 2011, at 8:45 AM, Peter Brinkmann wrote:
On Wed, Aug 31, 2011 at 6:25 AM, Dan Wilcox danomatika@gmail.com wrote:
I think it's much simpler to just add a call to get/set the message limit, say:
int libpd_max_message_length(); void libpd_set_max_message_length(int length);
This doesn't break any current code.
Having to set a custom limit each time is far more tedious then just setting it at startup.
Actually, breakage of current code is a feature as far as I am concerned because it makes people aware of the change, and it should be harmless because it's easy to fix. The language bindings for Java and Objective-C actually became simpler when I updated them for the new version.
I don't think the new signature of libpd_start_message is tedious, really. Essentially, I see two use cases: Either you know an a-priori limit on your message length, in which case there's the tiny extra effort of passing in the limit every time you start a message, or you don't have an a-priori limit, in which case you need to check the length before assembling a message anyway.
Another aspect is API design. One feature of a good API is that it's difficult to use incorrectly. With a separate call for setting the message limit, people will forget that the limit is a consideration. With the current solution, people will briefly contemplate the length of each message they start, which is a good thing.
... but you can simply return an error or print a message complaining when the message is too long. My whole point is that most people won't bother changing the limit and those that do will just pick a larger size with plenty of space anyway. It's too much work to bother setting it EACH and EVERY time. It's far LESS elegant, and dare I say intuitive. It seems like an unnecessary step. [eople that have problems will only run into this once, increase the max size, and then be fine. Why force them to compute a size manually each time when they could just be happily adding objects ... ?
Actually, I don't mean to make people compute the size every time The length parameter in libpd_start_message merely makes sure that you'll have enough space for that many elements; the number of elements in the actual message may be smaller. If you know that your messages will never exceed a certain length, then you can simply use that maximum length every time. In that situation, I would just wrap the call to libpd_start_message in a macro or a convenience function to get exactly the effect that you want.
By the way, there's also a third way between specifying the max size every time and creating a convenience function. The start message function will only allocate new memory if the requested size is larger than what was requested before. You can also say libpd_start_message(MAX_SIZE) right after you initialize libpd, and then start all messages with libpd_start_message(0), where 0 means that it'll just use whatever was allocated before. This is convenient, but it doesn't quite look right to me, so I'd probably still go with convenience functions. Peter
Okay, I just pushed the latest version of libpd (including dynamic allocation of the t_atom array for assembling list messages, i.e., no a-priori limit of the message size). I also updated the Android and iOS branches as well as the wiki. Cheers, Peter
On Tue, Aug 30, 2011 at 1:25 PM, Peter Brinkmann < peter.brinkmann@googlemail.com> wrote:
On Tue, Aug 30, 2011 at 12:51 PM, Mathieu Bouchard matju@artengine.cawrote:
On Tue, 30 Aug 2011, IOhannes m zmoelnig wrote:
why not simply resize the internal array as needed, starting with 32
elements and doubling whenever the limit is reached?
binbuf could be used for that, if the libc's realloc does in effect the same thing. i think that this is the case with modern glibc (Linux/CygWin) as well as Perl's allocator (when I looked at it 10 years ago), but not necessarily other libc implementations (FreeBSD/OSX, Google, Microsoft) that I didn't try it with, so if anyone could check, that would be interesting.
To test that, add millions of atoms to a binbuf and see how much time it takes. A «bad» implementation will roughly quadruple the time each time you double the number individual additions, whereas a «good» one will just double the time.
For the time being, I have something much simpler in mind: Just take the current call "int libpd_start_message(void)", which returns the current limit, and replace it with "int libpd_start_message(int length)", which takes a parameter indicating the length of the message and returns a nonzero error code if the length is too big.
I like this idea because it's almost as simple as the original approach, and the API agnostic with respect to the policy for handling long messages. The tentative implementation that I have right now still has the hard-coded limit of 32 elements, and libpd_start_message simply returns an error code when asked for a message with more than 32 elements. The next revision will dynamically resize the array if the requested length exceeds the current size, and it will only return an error code if the allocation of a larger array fails. Either way, there's no need to reallocate because the array contains nothing of interest when libpd_start_message is called, and so we can simply throw away the old array and allocate a new one.
The only disadvantage I see is that this doesn't sit well with the streaming-style C++ wrapper that Dan created for ofxPd, but I can think of workarounds for that. Cheers, Peter
On Sat, Aug 27, 2011 at 11:16 PM, Mathieu Bouchard matju@artengine.cawrote:
On Sun, 21 Aug 2011, Hans-Christoph Steiner wrote:
I think with the libpd API, you can write to Pd arrays. That's probably
you're best bet.
You must be meaning the pd API (m_pd.h).
There's nothing libpd-specific (z_libpd.h) about arrays.
That's not true. Recent versions of libpd come with functions for reading and writing arrays, memcpy-style.
It's a good thing, because IMHO, many of the functions whose name start with
the letters «libpd» are pointless, as they can already be done using typedmess() and such.
Okay, I'll bite.
Pointlessness is in the eye of the beholder. You may find many of the libpd wrappers pointless because you're working in C and you're already familiar with the functions in m_pd.h, but that's not the use case that I had in mind when writing libpd.
Can you bypass many of the functions in libpd and use m_pd.h directly? Sure, but then again maybe m_pd.h is pointless because you can just hack your binaries with a hex editor. That doesn't mean that that's a good level of abstraction to work at. libpd aims to provide a high-level API at a consistent level of abstraction, and I believe that that's the correct level of abstraction for the kind of work that libpd is intended for.
The immediate motivation was to create an API that would be easy to wrap for languages like Java and Python, but I also have deeper reasons for wanting to work at this level of abstraction. I'm hoping that we'll see a major redesign of Pd in the not too distant future. One goal we talked about at PdCon is to allow multiple instances of Pd. Another change I'm hoping to see is a rewrite that takes advantage of multiple cores on current machines. I also believe that such changes will be necessary to remain competitive.
The libpd API is meant to be (mostly) above such implementation details, while the low-level API will almost certainly change when Pd is updated. Pd itself will be much more nimble and maintainable if developers think about it at a higher level of abstraction.
In case you're interested, I recently added a few more functions to libpd, and I wrote up my reasoning behind them in a blog post: http://nettoyeur.noisepages.com/2011/08/pure-data-convention-libpd-and-a-min... Cheers, Peter
On Sun, 28 Aug 2011, Peter Brinkmann wrote:
Can you bypass many of the functions in libpd and use m_pd.h directly? Sure, but then again maybe m_pd.h is pointless because you can just hack your binaries with a hex editor. That doesn't mean that that's a good level of abstraction to work at. libpd aims to provide a high-level API at a consistent level of abstraction, and I believe that that's the correct level of abstraction for the kind of work that libpd is intended for.
Well, for the libpd message-passing, I felt like it added an API at the same level of abstraction as <m_pd.h>, except a tiny bit slower than SETFLOAT and SETSYMBOL macros, and which needed a bit more thread-safety than <m_pd.h>, as even the construction of the message has to be protected. Those are really small details, and to me, the biggie is that this API is not any easier than <m_pd.h>'s message passing to a C programmer.
The immediate motivation was to create an API that would be easy to wrap for languages like Java and Python, but I also have deeper reasons for wanting to work at this level of abstraction.
I don't see any practical difference in easiness of wrapping for other languages. I don't know how you see that.
I'm hoping that we'll see a major redesign of Pd in the not too distant future. One goal we talked about at PdCon is to allow multiple instances of Pd.
I don't see any planning about this in the way that the libpd api works, and I don't see how the libpd api currently helps for that.
Another change I'm hoping to see is a rewrite that takes advantage of multiple cores on current machines.
What's a «rewrite», and how much actual change do you wish for ? Do you have a plan for actual changes to the API ?
I also believe that such changes will be necessary to remain competitive.
If anyone really needs a big speed hike, then how about integrating SSE support in vanilla and/or libpd ? The prototype was made in 2005 or so, and then abandoned. That's a lot easier to do than to support double/triple/quad CPUs.
The libpd API is meant to be (mostly) above such implementation details, while the low-level API will almost certainly change when Pd is updated. Pd itself will be much more nimble and maintainable if developers think about it at a higher level of abstraction.
What constitutes a higher level of abstraction ?
BTW, yes, there are additions to your API that I hadn't seen, because I'm not using the latest version.
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
On Sun, Aug 28, 2011 at 1:26 PM, Mathieu Bouchard matju@artengine.cawrote:
On Sun, 28 Aug 2011, Peter Brinkmann wrote:
Can you bypass many of the functions in libpd and use m_pd.h directly?
Sure, but then again maybe m_pd.h is pointless because you can just hack your binaries with a hex editor. That doesn't mean that that's a good level of abstraction to work at. libpd aims to provide a high-level API at a consistent level of abstraction, and I believe that that's the correct level of abstraction for the kind of work that libpd is intended for.
Well, for the libpd message-passing, I felt like it added an API at the same level of abstraction as <m_pd.h>, except a tiny bit slower than SETFLOAT and SETSYMBOL macros, and which needed a bit more thread-safety than <m_pd.h>, as even the construction of the message has to be protected. Those are really small details, and to me, the biggie is that this API is not any easier than <m_pd.h>'s message passing to a C programmer.
Matter of taste, I suppose. When working with libpd, I want to think in terms of calls to libpd. Reaching into m_pd.h is a context switch that I'd rather avoid, and with the latest revision of libpd I can avoid it altogether. And I do think that saying "libpd_get_symbol(a)" is simpler than saying "a.a_w.w_symbol->s_name" (although the former is just a macro that maps to the latter).
Then again, my instincts run libertarian. I have no desire to tell you how to do your work. When you include z_libpd.h, you get m_pd.h with it. If you want to work too hard, I won't stand in your way;)
The immediate motivation was to create an API that would be easy to wrap
for languages like Java and Python, but I also have deeper reasons for wanting to work at this level of abstraction.
I don't see any practical difference in easiness of wrapping for other languages. I don't know how you see that.
One major simplification is the use of built-in data types vs custom structs and unions. Many of the functions that you find pointless basically do two things; they convert const char* to t_symbol and then delegate to functions in m_pd.h. With this transition to built-in data types, you can simply run SWIG on the header file and automatically create bindings for a significant part of libpd for lots of target languages; only the callbacks will require some additional work. Without this conversion, you would have to descend into the netherworld of custom typemaps and such.
I'm hoping that we'll see a major redesign of Pd in the not too distant
future. One goal we talked about at PdCon is to allow multiple instances of Pd.
I don't see any planning about this in the way that the libpd api works, and
I don't see how the libpd api currently helps for that.
It doesn't. The point is that it would be easy to extend because you'd just need to add an instance pointer to the parameter list of a smallish collection of functions.
Incidentally, I've also gotten flak for not baking any multi-instance support into the initial version of libpd (I did consider it, but it seemed pointless because with the current version of Pd it would still have just one single instance). I figure that as long as I'm drawing fire from both low-level C hackers and high-level OOP types, I must be doing something right;)
Another change I'm hoping to see is a rewrite that takes advantage of
multiple cores on current machines.
What's a «rewrite», and how much actual change do you wish for ? Do you have a plan for actual changes to the API ?
It's more of a general consideration, and I don't have any road map in mind. Right now I'm just hoping to have a conversation about this.
I also believe that such changes will be necessary to remain competitive.
If anyone really needs a big speed hike, then how about integrating SSE support in vanilla and/or libpd ? The prototype was made in 2005 or so, and then abandoned. That's a lot easier to do than to support double/triple/quad CPUs.
I'd be more than happy to see that, ideally in vanilla because I want libpd to remain nothing more than a thin wrapper on top of Pd.
The libpd API is meant to be (mostly) above such implementation details,
while the low-level API will almost certainly change when Pd is updated. Pd itself will be much more nimble and maintainable if developers think about it at a higher level of abstraction.
What constitutes a higher level of abstraction ?
In this case, I mostly mean hiding implementation details. For example, as an application programmer I don't want to know what exactly is going on inside an instance of t_atom, I just want to get string or float values in and out. Cheers, Peter
On Sun, 28 Aug 2011, Peter Brinkmann wrote:
One major simplification is the use of built-in data types vs custom structs and unions.
You mean you simplify by making things more low-level ?
const char * is rarely ever called high-level...
With this transition to built-in data types, you can simply run SWIG on the header file and automatically create bindings for a significant part of libpd for lots of target languages; only the callbacks will require some additional work. Without this conversion, you would have to descend into the netherworld of custom typemaps and such.
When you are using typemaps, do you have the impression that you're using SWIG in a high-level way ? Isn't the lack of typemaps a sign that your API isn't very abstract ?
Incidentally, I've also gotten flak for not baking any multi-instance support into the initial version of libpd (I did consider it, but it seemed pointless because with the current version of Pd it would still have just one single instance).
The point would be to make all the backward-compat and forward-compat you need for supporting a future version of pd that will include features that you're already thinking about.
But you don't have to add an argument everywhere. I bet that you'll add a libpd_set_current_pd_instance(t_pd_interpreter *) function that will set a global variable used by the rest of the pd api. That would be consistent with the stateful message-passing in many small steps using a hidden global array. Later you can make all of that thread-safe by making all of those functions call pthread_self() to figure out which thread they're in, and replace the globals by threadwise-locals.
I figure that as long as I'm drawing fire from both low-level C hackers and high-level OOP types, I must be doing something right;)
It's bad to insist on a low-level vs high-level dichotomy. It's been quite a while that those words cause confusion because they give the impression that those levels are all stacked in top of each other along one axis of highness. The truth is a lot more complicated, in which levels aren't well-defined, more abstract isn't necessarily better, more concrete isn't necessarily better either, it depends a lot on what you want to achieve, and what you expect for the future, etc. ; I still use expressions like low-level or high-level sometimes, but it isn't as seriously as when people first invented the term.
So, it's possible that you're doing something right, but drawing fire from people that you categorise in two bins is probably not a good sign of what you think it is.
It's more of a general consideration, and I don't have any road map in mind. Right now I'm just hoping to have a conversation about this.
Well, does that involve multiple interpreters, with one interpreter per core, but still in the same process, with a way to pass messages quickly from one interpreter to the other ?
Would they be sharing the same gensym, or would they need to re-gensym everything when talking from one interpreter to the other ?
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
On Sun, Aug 28, 2011 at 5:04 PM, Mathieu Bouchard matju@artengine.cawrote:
On Sun, 28 Aug 2011, Peter Brinkmann wrote:
One major simplification is the use of built-in data types vs custom
structs and unions.
You mean you simplify by making things more low-level ?
const char * is rarely ever called high-level...
Well, I'm taking a complicated implementation detail (t_symbol) and I'm providing an API that lets application developers to refer to this detail by name, without having to think about the data structure behind it. I consider that an abstraction, even if the name is given as const char *.
Incidentally, I've also gotten flak for not baking any multi-instance
support into the initial version of libpd (I did consider it, but it seemed pointless because with the current version of Pd it would still have just one single instance).
The point would be to make all the backward-compat and forward-compat you need for supporting a future version of pd that will include features that you're already thinking about.
Nah, that would have been a clear case of overdesign, preparing for some future development that may never happen. We'll figure out how to support multiple instances when multiple instances become a possibility.
But you don't have to add an argument everywhere. I bet that you'll add a libpd_set_current_pd_instance(**t_pd_interpreter *) function that will set a global variable used by the rest of the pd api. That would be consistent with the stateful message-passing in many small steps using a hidden global array. Later you can make all of that thread-safe by making all of those functions call pthread_self() to figure out which thread they're in, and replace the globals by threadwise-locals.
I thought I had already explained this when we had our little chat over at pd-everywhere, but I'll try again. Calling the message assembly API of libpd stateful is technically true but completely misleading because the hidden state is only meant to be used in a very specific and limited way. Here's the problem that it is supposed to solve: You want to translate a heterogeneous list of objects in Java into an array of type t_atom in C. That's all.
Doing the entire conversion in JNI would be a huge pain, and dynamically allocating t_atom arrays in JNI would be a pain, too. So, I chose to allocate one array up front and then provide a set of functions that will populate the array with values, in a way that's easy to wrap for Java. The fact that it's a global array does not cause any problems because the entire calling sequence needs to be protected by a lock, and so there is only one method that will ever access the array at one time. The lock would be required in any case, because any access to the symbol table needs to be locked.
The upshot is, the array you mention really acts as a local variable for a couple of methods in languages like Java or Python or Objective-C. The fact that it's currently a hidden global variable is an implementation detail that does not add problematic global state to libpd. (I can't even think of a way to creatively misuse this mechanism to introduce global statue through the back door.) Believe me, I've agonized over this, but it's been working very nicely for more than a year now, and I still can't think of a simpler way to assemble compound messages in Java.
It's more of a general consideration, and I don't have any road map in
mind. Right now I'm just hoping to have a conversation about this.
Well, does that involve multiple interpreters, with one interpreter per core, but still in the same process, with a way to pass messages quickly from one interpreter to the other?
That's not at all what I have in mind. What I'm really thinking about is one interpreter that does a topological sort on the signal processing chain and then parallelizes the computation on the fly, but that's a topic for another day. I'm outta here. Cheers, Peter
On Sun, 28 Aug 2011, Peter Brinkmann wrote:
Well, I'm taking a complicated implementation detail (t_symbol)
but t_symbol is a higher-level structure for wrapping the const char *. :}
I thought I had already explained this when we had our little chat over at pd-everywhere, but I'll try again. Calling the message assembly API of libpd stateful is technically true but completely misleading because the hidden state is only meant to be used in a very specific and limited way.
Hidden states usually are meant to be used in very specific and limited ways... I don't know at all the distinction you're trying to make. The statefulness of libpd is not misleading. Trying to say that it's not really stateful, is misleading. I don't know what kind of connotations the statefulness means to you and why you're trying to avoid calling it as such.
Anyway, it's not a big loss to have statefulness in those circumstances, as it's just before the beginning of a part that would have to be locked anyway, or in a part that would have to be locked anyway (if calling gensym).
Here's the problem that it is supposed to solve: You want to translate a heterogeneous list of objects in Java into an array of type t_atom in C. That's all.
Btw, did you look at Pascal Gauthier's library ?
...
and also, I just read your libpd_read_array and libpd_write_array functions. They don't work in 64-bit mode, in which sizeof(t_word) != sizeof(t_float).
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
Here's the problem that it is supposed to solve: You want to translate a
heterogeneous list of objects in Java into an array of type t_atom in C. That's all.
Btw, did you look at Pascal Gauthier's library ?
Yes, but I think it solves a different problem. In particular, I don't want to mirror t_atom on the Java side.
and also, I just read your libpd_read_array and libpd_write_array functions. They don't work in 64-bit mode, in which sizeof(t_word) != sizeof(t_float).
Yikes, good point. I'll look into it. Peter
On Mon, Aug 29, 2011 at 1:15 AM, Mathieu Bouchard matju@artengine.cawrote:
[...]
and also, I just read your libpd_read_array and libpd_write_array functions.
They don't work in 64-bit mode, in which sizeof(t_word) != sizeof(t_float).
Okay, should be fixed now. I just pushed the latest revision to Gitorious. Peter
On Sun, 28 Aug 2011, Peter Brinkmann wrote:
Pointlessness is in the eye of the beholder. You may find many of the libpd wrappers pointless because you're working in C and you're already familiar with the functions in m_pd.h,
Well, actually, most of the time, I've been using my own C++ wrapper (including a custom preprocessor) so that I can avoid much of <m_pd.h>, but more recently I've been working outside of that context.
The difference in conciseness is very large.
| Mathieu Bouchard ---- tél: +1.514.383.3801 ---- Villeray, Montréal, QC
-
Dan Wilcox -
Hans-Christoph Steiner -
IOhannes m zmoelnig -
Jonathan Wilkes -
Mathieu Bouchard -
Peter Brinkmann -
ronni montoya