Hey Martin and all,
Just had a thought: originally everything in the 'mrpeach' folder was bundled into a single library, which I think Martin didn't really intend. I did it to get Martin's valuable code out there in a kind of beta way. Now I think its quite clear that the 'net' and 'osc' sections in 'mrpeach' are really the canonical way of doing networking and OSC with Pd, so I think it makes sense to have them distributed as the 'net' and 'osc' library. I'm guessing Martin had that intention originally. Also, I noticed that IOhannes's filed an Intent-To- Package for something called 'pd-osc', which I am guessing is externals/mrpeach/osc.
So my only question is: what becomes of 'mrpeach'?
.hc
----------------------------------------------------------------------------
News is what people want to keep hidden and everything else is publicity. - Bill Moyers
On 2010-11-28 12:13, Hans-Christoph Steiner wrote:
Hey Martin and all,
Just had a thought: originally everything in the 'mrpeach' folder was bundled into a single library, which I think Martin didn't really intend. I did it to get Martin's valuable code out there in a kind of beta way. Now I think its quite clear that the 'net' and 'osc' sections in 'mrpeach' are really the canonical way of doing networking and OSC with Pd, so I think it makes sense to have them distributed as the 'net' and 'osc' library. I'm guessing Martin had that intention originally. Also, I noticed that IOhannes's filed an Intent-To-Package for something called 'pd-osc', which I am guessing is externals/mrpeach/osc.
So my only question is: what becomes of 'mrpeach'?
I agree it makes sense to package things by function rather than by author. So here's my take:
Inside OSC ATM is [pipelist], which should maybe go into some list lib although it's useful for delaying timestamped OSC messages.
In Net, [tcpreceive] is unnecessary as its functionality is covered by [tcpclient].
[binfile] and [midifile] should go into a file handling lib.
[slipenc] and [slipdec] are useful for OSC over serial, so maybe put them in Net?
[tab2flist], [flist2tab], [tabfind] operate on table/arrays.
[sqosc~], [rc~], [rcosc~], [rojo~] are audio generators.
[op~] is similar to [>~], [<~] etc but doesn't have trouble instantiating without hexloader.
str only works with the (unaccepted) string patch, so it could be dropped.
cmos is not really useful except to simulate some CMOS logic ICs.
[life2x] is a cellular automaton.
[which] tends to crash a lot, but can help to find the path to an external.
[runningmean] is a math object.
Martin
On Sun, 2010-11-28 at 13:38 -0500, Martin Peach wrote:
On 2010-11-28 12:13, Hans-Christoph Steiner wrote:
Hey Martin and all,
Just had a thought: originally everything in the 'mrpeach' folder was bundled into a single library, which I think Martin didn't really intend. I did it to get Martin's valuable code out there in a kind of beta way. Now I think its quite clear that the 'net' and 'osc' sections in 'mrpeach' are really the canonical way of doing networking and OSC with Pd
I'd hesitate to call using mrpeach/net the 'canonical' way. Last time I checked, there were still issues with many classes, in particular the blocking issue of [tcpsend]/[tcpclient]/[tcpserver] discussed in a plethora of mails. That's also the reason why IOhannes rewrote those and released them as the iemnet library. The classes from iemnet are high-performance and don't suffer from any blocking issue.
, so I think it makes sense to have them distributed as the 'net'
and 'osc' library. I'm guessing Martin had that intention originally. Also, I noticed that IOhannes's filed an Intent-To-Package for something called 'pd-osc', which I am guessing is externals/mrpeach/osc.
So my only question is: what becomes of 'mrpeach'?
[slipenc] and [slipdec] are useful for OSC over serial, so maybe put them in Net?
Yeah, and also they are very useful for transporting message oriented protocols over TCP. OSC 1.1 even uses SLIP for sending OSC over TCP (Which reminds me, that I should update [unpackOSCstream] and [packOSCstream] to use SLIP instead of the OSC 1.0 message length prefix).
Roman
On 2010-11-28 15:57, Roman Haefeli wrote:
On Sun, 2010-11-28 at 13:38 -0500, Martin Peach wrote:
On 2010-11-28 12:13, Hans-Christoph Steiner wrote:
Hey Martin and all,
Just had a thought: originally everything in the 'mrpeach' folder was bundled into a single library, which I think Martin didn't really intend. I did it to get Martin's valuable code out there in a kind of beta way. Now I think its quite clear that the 'net' and 'osc' sections in 'mrpeach' are really the canonical way of doing networking and OSC with Pd
I'd hesitate to call using mrpeach/net the 'canonical' way. Last time I checked, there were still issues with many classes, in particular the blocking issue of [tcpsend]/[tcpclient]/[tcpserver] discussed in a plethora of mails. That's also the reason why IOhannes rewrote those and released them as the iemnet library. The classes from iemnet are high-performance and don't suffer from any blocking issue.
That's another reason they should be in net instead of mrpeach: it seems that having my name on the folder inhibits others from improving the objects, so we end up with multiple parallel incompatible objects, in this case with the same names.
(And when was the last time you checked?)
Martin
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 11/28/2010 10:17 PM, Martin Peach wrote:
On 2010-11-28 15:57, Roman Haefeli wrote:
That's another reason they should be in net instead of mrpeach: it seems that having my name on the folder inhibits others from improving the objects, so we end up with multiple parallel incompatible objects, in this case with the same names.
a) my intention when writing iemnet was to maintain compatibility with mrpeachnet (with silently dropping things which i find either ugly hackz to circumvent problems in the current implementation (e.g the "unblock" message) or things that have nothing todo with networking (the possibility to directly send the contents of files)
in most cases they should thus be exchangeablee
b) my intention as a debian packager was to provide both packages as "pd-iemnet" and "pd-mrpeachnet", not making any of those the "canonical" library. i still think thaat both share sufficient parts of their interface to make them be called "compatible", so i would also make both of them provide "pd-net", allowing people to use them as "net/udpsend" (if they can stick to the common API) or as "mrpeach/net/udpsend" if they cannot live without sending entire files directly.
i haven't done the actual packaging yet (given my currently available time...)
mfgasdr IOhannes
PS: and yes, i agree that the packages should be grouped by functionality.
On Nov 29, 2010, at 4:36 AM, IOhannes m zmölnig wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 11/28/2010 10:17 PM, Martin Peach wrote:
On 2010-11-28 15:57, Roman Haefeli wrote:
That's another reason they should be in net instead of mrpeach: it seems that having my name on the folder inhibits others from improving the objects, so we end up with multiple parallel incompatible objects, in this case with the same names.
a) my intention when writing iemnet was to maintain compatibility with mrpeachnet (with silently dropping things which i find either ugly hackz to circumvent problems in the current implementation (e.g the "unblock" message) or things that have nothing todo with networking (the possibility to directly send the contents of files)
in most cases they should thus be exchangeablee
b) my intention as a debian packager was to provide both packages as "pd-iemnet" and "pd-mrpeachnet", not making any of those the "canonical" library. i still think thaat both share sufficient parts of their interface to make them be called "compatible", so i would also make both of them provide "pd-net", allowing people to use them as "net/udpsend" (if they can stick to the common API) or as "mrpeach/net/udpsend" if they cannot live without sending entire files directly.
i haven't done the actual packaging yet (given my currently available time...)
It is OK IMHO to have iemnet use the names as the mrpeach objects as long as the stated goal is interface compatibility. But I think this packaging idea is just added complication without and real benefit. A library is a library in Debian from what I have seen. I've never seen this kind of arrangement for libraries in Debian, so I think it would be a bad idea to start it with Pd.
.hc
mfgasdr IOhannes
PS: and yes, i agree that the packages should be grouped by functionality. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAkzzdBMACgkQkX2Xpv6ydvRAcACgh5rCtzrMvXKXpVQMaiTTwU2M GXgAn12FQsOYbvME/5gbbnPBwZvRpf+a =4uOC -----END PGP SIGNATURE-----
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
----------------------------------------------------------------------------
"It is convenient to imagine a power beyond us because that means we don't have to examine our own lives.", from "The Idols of Environmentalism", by Curtis White
On 2010-11-29 18:36, Hans-Christoph Steiner wrote:
It is OK IMHO to have iemnet use the names as the mrpeach objects as long as the stated goal is interface compatibility. But I think this packaging idea is just added complication without and real benefit. A library is a library in Debian from what I have seen. I've never seen this kind of arrangement for libraries in Debian, so I think it would be a bad idea to start it with Pd.
i mean something like libGL; this has been in debian for a while.
b,sdr IOhannes
On Nov 29, 2010, at 12:48 PM, IOhannes m zmoelnig wrote:
On 2010-11-29 18:36, Hans-Christoph Steiner wrote:
It is OK IMHO to have iemnet use the names as the mrpeach objects as long as the stated goal is interface compatibility. But I think this packaging idea is just added complication without and real benefit. A library is a library in Debian from what I have seen. I've never seen this kind of arrangement for libraries in Debian, so I think it would be a bad idea to start it with Pd.
i mean something like libGL; this has been in debian for a while.
b,sdr IOhannes
Sure that makes sense if you are really planning on keeping iemnet in sync with and compatible with mrpeach's net. It doesn't seem worth the hassle here at all, and will add confusion.
.hc
----------------------------------------------------------------------------
I have always wished for my computer to be as easy to use as my telephone; my wish has come true because I can no longer figure out how to use my telephone." --Bjarne Stroustrup (creator of C++)
On Sun, 2010-11-28 at 16:17 -0500, Martin Peach wrote:
On 2010-11-28 15:57, Roman Haefeli wrote:
On Sun, 2010-11-28 at 13:38 -0500, Martin Peach wrote:
On 2010-11-28 12:13, Hans-Christoph Steiner wrote:
Hey Martin and all,
Just had a thought: originally everything in the 'mrpeach' folder was bundled into a single library, which I think Martin didn't really intend. I did it to get Martin's valuable code out there in a kind of beta way. Now I think its quite clear that the 'net' and 'osc' sections in 'mrpeach' are really the canonical way of doing networking and OSC with Pd
I'd hesitate to call using mrpeach/net the 'canonical' way. Last time I checked, there were still issues with many classes, in particular the blocking issue of [tcpsend]/[tcpclient]/[tcpserver] discussed in a plethora of mails. That's also the reason why IOhannes rewrote those and released them as the iemnet library. The classes from iemnet are high-performance and don't suffer from any blocking issue.
That's another reason they should be in net instead of mrpeach: it seems that having my name on the folder inhibits others from improving the objects, so we end up with multiple parallel incompatible objects, in this case with the same names.
(And when was the last time you checked?)
Martin
I call mrpeach/net canonical not because I believe is it perfect and bugfree, but rather because it is the established, proven way of doing more elaborate networking. Its the best option out there. iemnet is just a fork of that with some specific changes. iemnet is very new and not tested as much, so it seems a really bad idea to start basing things off of it, like how to package things in Debian, etc. Who knows, perhaps mrpeach/net and iemnet will merge again.
.hc
On 2010-12-01 03:02, Hans-Christoph Steiner wrote:
I call mrpeach/net canonical not because I believe is it perfect and bugfree, but rather because it is the established, proven way of doing more elaborate networking.
i think it is not canonical as it tends to change it's API to workaround certain problems.
Its the best option out there. iemnet is just a fork of that with some specific changes. iemnet is very new and not tested as much, so it seems a really bad idea to start basing things off of it,
i'd rather not read the above.
anyhow, i definitely wouldn't call iemnet the "canonical". i think canonicity should be more about the object's API than about the implementation itself.
like how to package things in Debian, etc. Who knows, perhaps mrpeach/net and iemnet will merge again.
that is desireable from my pov (but there is still some work todo on the iemnet part, which i would like to be not hindered by legacy code)
fgmasdr IOhannes
On Tue, 2010-11-30 at 21:02 -0500, Hans-Christoph Steiner wrote:
On Sun, 2010-11-28 at 16:17 -0500, Martin Peach wrote:
On 2010-11-28 15:57, Roman Haefeli wrote:
On Sun, 2010-11-28 at 13:38 -0500, Martin Peach wrote:
On 2010-11-28 12:13, Hans-Christoph Steiner wrote:
Hey Martin and all,
Just had a thought: originally everything in the 'mrpeach' folder was bundled into a single library, which I think Martin didn't really intend. I did it to get Martin's valuable code out there in a kind of beta way. Now I think its quite clear that the 'net' and 'osc' sections in 'mrpeach' are really the canonical way of doing networking and OSC with Pd
I'd hesitate to call using mrpeach/net the 'canonical' way. Last time I checked, there were still issues with many classes, in particular the blocking issue of [tcpsend]/[tcpclient]/[tcpserver] discussed in a plethora of mails. That's also the reason why IOhannes rewrote those and released them as the iemnet library. The classes from iemnet are high-performance and don't suffer from any blocking issue.
That's another reason they should be in net instead of mrpeach: it seems that having my name on the folder inhibits others from improving the objects, so we end up with multiple parallel incompatible objects, in this case with the same names.
(And when was the last time you checked?)
Martin
I call mrpeach/net canonical not because I believe is it perfect and bugfree, but rather because it is the established, proven way of doing more elaborate networking.
Probably it can be considered established because it has been there for quite some time, nevertheless I've found it hard to use for what I would call 'elaborate' networking right from the beginning. I don't know in what scenario you think it might have an established use. Actually, many protocols that I can think of would be fun to use with [tcpserver] are packet oriented (as opposed to stream oriented). And since TCP is a stream oriented protocol, there is no way [tcpserver] can be directly used as is. Rather one has to make sure while using a packet oriented protocol to correctly create protocol compliant packets from what is received from [tcpserver] from several sockets. It might well be that parts of different packets are received interleaved. Furthermore, the creation iemnet's [tcpserver] was actually driven by issues [mrpeach/tcpserver] is still suffering from (Check my other mail). I hope I could challenge what you call 'established', 'proven' and 'elaborate'.
Its the best option out there. iemnet is just a fork of that with some specific changes. iemnet is very new and not tested as much,
Believe me, it's very well tested.
so it seems a really bad idea to start basing things off of it, like how to package things in Debian, etc. Who knows, perhaps mrpeach/net and iemnet will merge again.
Actually, I'd hope so. From what I can tell, they can be exchanged transparently with the following exceptions:
* The output of the fifth outlet in [tcpserver] is different in iemnet. Since the fifth outlet has been added quite recently in mrpeach and also has changed since, I don't think that would be much of an issue.
* iemnet's object classes don't support sending from file. This is actually a feature not related to networking. Also it can be simply added by patching by using mrpeach's [binfile].
* iemnet's object classes add the port number to the address outlet (which doesn't break mrpeach)
* some performance related settings/methods were removed from iemnet's [tcpserver], simply because they are not necessary (no buffer overflows nor blocking of Pd ever happened with iemnet's [tcpserver] in my tests).
On a side-note: the netpd-server-tcpserver.pd patch mentioned in my earlier post runs with both, mrpeach's and iemnet's [tcpserver]. No other changes are necessary when switching.
Roman
On Sun, 2010-11-28 at 16:17 -0500, Martin Peach wrote:
On 2010-11-28 15:57, Roman Haefeli wrote:
On Sun, 2010-11-28 at 13:38 -0500, Martin Peach wrote:
On 2010-11-28 12:13, Hans-Christoph Steiner wrote:
Hey Martin and all,
Just had a thought: originally everything in the 'mrpeach' folder was bundled into a single library, which I think Martin didn't really intend. I did it to get Martin's valuable code out there in a kind of beta way. Now I think its quite clear that the 'net' and 'osc' sections in 'mrpeach' are really the canonical way of doing networking and OSC with Pd
I'd hesitate to call using mrpeach/net the 'canonical' way. Last time I checked, there were still issues with many classes, in particular the blocking issue of [tcpsend]/[tcpclient]/[tcpserver] discussed in a plethora of mails. That's also the reason why IOhannes rewrote those and released them as the iemnet library. The classes from iemnet are high-performance and don't suffer from any blocking issue.
That's another reason they should be in net instead of mrpeach: it seems that having my name on the folder inhibits others from improving the objects, so we end up with multiple parallel incompatible objects, in this case with the same names.
(And when was the last time you checked?)
Today. I made a little stresstest patch for the netpd-server based on [tcpserver]. Since I currently don't have a second computer at hand, it's difficult for me to test for the blocking issue, but I found another issue. The stresstest client patch sends 1000 messages per second prefixed with its own socket number, so that the server sends them back to the client. The client checks, if the message are returned in order and also if they are valid/have the same content as initially sent. It turns out, that although the messages are received in order on the server, the client seems not to receive some while getting duplicates of others. Check those logs:
How they are received on the server from the client and how they are sent back to the client:
print: 6 test 523 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: 6 test 524 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: 6 test 525 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: 6 test 526 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: 6 test 527 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: 6 test 528 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: 6 test 529 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: 6 test 530 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: 6 test 531 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: 6 test 532 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: 6 test 533 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn
How they are received by the client:
print: test 523 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: test 525 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: test 526 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: test 527 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: test 528 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: test 529 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: test 529 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: test 529 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: test 529 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: test 529 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: test 529 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: test 531 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: test 533 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn print: test 533 eins zwei drei vier fuenf sechs sieben acht neun zehn elf zwoelf dreizehn vierzehn fuenfzehn sechszehn siebzehn
The situation significantly improves when lowering the rate the messages are sent. However, in an application like netpd it's hard to control the overall message rate beforehand.
The same stresstest performed with [iemnet/tcpserver] instead of [mrpeach/tcpserver] doesn't show any irregularities, even with 5 instances of the stresstest patch running as clients (more can my CPU not handle | The high CPU load comes from the [tcpserver] object, but from the fact that FUDI delimiting is done in the byte realm with Pd objects).
You find all the patches (netpd-server-tcpserver.pd and stresstest.pd) on: https://github.com/reduzent/netpd-server
Roman
-
Hans-Christoph Steiner -
IOhannes m zmoelnig -
IOhannes m zmölnig
-
Martin Peach -
Roman Haefeli