Hi all,
Apologies for the rather esoteric question.
I'm trying to send OSC to/from Pd over TCP using the liblo OSC library and the mrpeach/ OSC externals [tcpserver], [packOSCstream] and [unpackOSCstream]. I've attached my testing patch.
I am able to connect to [tcpserver] from liblo, and receive raw bytes. However, [unpackOSCstream] seems unable to unpack the data. It just outputs nothing for the bytes received.
Likewise, if I send to liblo from [tcpserver] using the |client( selector, liblo crashes with the message "error? message too short?".
So, it seems like [packOSCstream] and [unpackOSCstream] are (un)packing message in a format that can't be used with external clients/servers. Is that correct?
Has anyone else managed bi-directional communication using OSC over TCP with an external application?
Thanks,
Jamie
hey jamie,
Yep, in this project (https://vimeo.com/24818342) it was going back and forth from the core app (cpp) to pd. Using OSC. Cannot recall if TCP or not, but can probably track it down. (that should not make much of a difference at least outside the network layer).
So you send from liblo and pd receives, but no data unpacks? Did I get it write? If you open a ./oscdump on command line you get the data? (meaning it would be a pd-side issue...)
Best, pedro
On Fri, Nov 9, 2012 at 2:51 PM, Jamie Bullock jamie@jamiebullock.comwrote:
Hi all,
Apologies for the rather esoteric question.
I'm trying to send OSC to/from Pd over TCP using the liblo OSC library and the mrpeach/ OSC externals [tcpserver], [packOSCstream] and [unpackOSCstream]. I've attached my testing patch.
I am able to connect to [tcpserver] from liblo, and receive raw bytes. However, [unpackOSCstream] seems unable to unpack the data. It just outputs nothing for the bytes received.
Likewise, if I send to liblo from [tcpserver] using the |client( selector, liblo crashes with the message "error? message too short?".
So, it seems like [packOSCstream] and [unpackOSCstream] are (un)packing message in a format that can't be used with external clients/servers. Is that correct?
Has anyone else managed bi-directional communication using OSC over TCP with an external application?
Thanks,
Jamie
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
Hi Pedro,
Thanks for the reply. To clarify I can OSC over *UDP* from liblo to/from Pd just fine using the mrpeach externals. It's using TCP, where I run into trouble. I think it's due to the streamed nature of TCP, and message boundaries, but I am not an expert on this.
On 9 Nov 2012, at 14:00, Pedro Lopes pedro.lopes@ist.utl.pt wrote:
hey jamie,
Yep, in this project (https://vimeo.com/24818342) it was going back and forth from the core app (cpp) to pd. Using OSC. Cannot recall if TCP or not, but can probably track it down. (that should not make much of a difference at least outside the network layer).
So you send from liblo and pd receives, but no data unpacks? Did I get it write? If you open a ./oscdump on command line you get the data? (meaning it would be a pd-side issue...)
Yes that's correct. There is an example "echo server" that comes with liblo. I can get bidirectional communication liblo <-> using this example. If I replace one end of the communication with [tcpserver], liblo can connect, [tcpserver] receives data, but [unpackOSCstream] can't seem to unpack the data. Likewise [packOSCstream] doesn't seem to generate a data format that liblo can understand.
best,
Jamie
I just digged up my thesis (funny how memory works) and I was using UDP. So its TCP, so you can do liblo to liblo? (or for that matter processing using oscp5, which is fast to test). If so, could be something with pd objects.
On Fri, Nov 9, 2012 at 4:19 PM, Jamie Bullock jamie@jamiebullock.comwrote:
Hi Pedro,
Thanks for the reply. To clarify I can OSC over *UDP* from liblo to/from Pd just fine using the mrpeach externals. It's using TCP, where I run into trouble. I think it's due to the streamed nature of TCP, and message boundaries, but I am not an expert on this.
On 9 Nov 2012, at 14:00, Pedro Lopes pedro.lopes@ist.utl.pt wrote:
hey jamie,
Yep, in this project (https://vimeo.com/24818342) it was going back and
forth from the core app (cpp) to pd. Using OSC. Cannot recall if TCP or not, but can probably track it down. (that should not make much of a difference at least outside the network layer).
So you send from liblo and pd receives, but no data unpacks? Did I get
it write? If you open a ./oscdump on command line you get the data? (meaning it would be a pd-side issue...)
Yes that's correct. There is an example "echo server" that comes with liblo. I can get bidirectional communication liblo <-> using this example. If I replace one end of the communication with [tcpserver], liblo can connect, [tcpserver] receives data, but [unpackOSCstream] can't seem to unpack the data. Likewise [packOSCstream] doesn't seem to generate a data format that liblo can understand.
best,
Jamie
If you look inside [unpackOSCstream] you'll see that it uses [slipdec]. So the other end must be using [slipenc] or they won't match up. Unless you are doing this over a serial connection there is no need to use SLIP at all. [unpackOSC] should work properly with [tcpclient] or [tcpserver].
Martin
From: jamie@jamiebullock.com Date: Fri, 9 Nov 2012 15:19:07 +0000 To: pedro.lopes@ist.utl.pt CC: pd-list@iem.at; martin.peach@sympatico.ca Subject: Re: [PD] bi-directional OSC over TCP from liblo
Hi Pedro,
Thanks for the reply. To clarify I can OSC over *UDP* from liblo to/from Pd just fine using the mrpeach externals. It's using TCP, where I run into trouble. I think it's due to the streamed nature of TCP, and message boundaries, but I am not an expert on this.
On 9 Nov 2012, at 14:00, Pedro Lopes pedro.lopes@ist.utl.pt wrote:
hey jamie,
Yep, in this project (https://vimeo.com/24818342) it was going back and forth from the core app (cpp) to pd. Using OSC. Cannot recall if TCP or not, but can probably track it down. (that should not make much of a difference at least outside the network layer).
So you send from liblo and pd receives, but no data unpacks? Did I get it write? If you open a ./oscdump on command line you get the data? (meaning it would be a pd-side issue...)
Yes that's correct. There is an example "echo server" that comes with liblo. I can get bidirectional communication liblo <-> using this example. If I replace one end of the communication with [tcpserver], liblo can connect, [tcpserver] receives data, but [unpackOSCstream] can't seem to unpack the data. Likewise [packOSCstream] doesn't seem to generate a data format that liblo can understand.
best,
Jamie
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list
On 11/09/2012 05:14 PM, martin.peach@sympatico.ca wrote:
If you look inside [unpackOSCstream] you'll see that it uses [slipdec]. So the other end must be using [slipenc] or they won't match up. Unless you are doing this over a serial connection there is no need to use SLIP at all. [unpackOSC] should work properly with [tcpclient] or [tcpserver].
but TCP/IP *is* a serial connection (that is: stream-based rather than packet-based). if you are sending OSC-data over TCP/IP without using SLIP, you are violating the OSC1.1 standard.
fgmasdr IOhannes
Date: Fri, 9 Nov 2012 17:25:41 +0100 From: zmoelnig@iem.at To: pd-list@iem.at Subject: Re: [PD] bi-directional OSC over TCP from liblo
On 11/09/2012 05:14 PM, martin.peach@sympatico.ca wrote:
If you look inside [unpackOSCstream] you'll see that it uses [slipdec]. So the other end must be using [slipenc] or they won't match up. Unless you are doing this over a serial connection there is no need to use SLIP at all. [unpackOSC] should work properly with [tcpclient] or [tcpserver].
but TCP/IP *is* a serial connection (that is: stream-based rather than packet-based). if you are sending OSC-data over TCP/IP without using SLIP, you are violating the OSC1.1 standard.
Ah yes. I was mixed up with the original use of slip: to pack entire tcp/ip packets, not just the payload, for transmission over a byte-oriented serial link.
Martin
On 11/09/2012 02:51 PM, Jamie Bullock wrote:
So, it seems like [packOSCstream] and [unpackOSCstream] are (un)packing message in a format that can't be used with external clients/servers. Is that correct?
from your experience with liblo, you seem to conclude that: a) all "external clients/servers" are build on top of liblo. b) liblo handles OSC over TCP/IP correctly c) [(un)packOSCstream] handles it wrongly.
anyhow: originally OSC did not define how to transmit OSC-packets over a stream-based protocol like TCP/IP (that has no notion of ending packets). implementations that wanted to transmit OSC over stream-based protocols had to find solutions for themselves.
a few years ago, the OSC specs have been extended, to explicitely mention "SLIP" encoding as the means to packetize streams of OSC-data.
[packOSCstream] implements this standard by use of [slipenc].
i don't know, how liblo packetizes the data for tcp/ip, but if it is no t SLIP, then this is a bug in liblo (if you want a "standard" behaviour). if it *uses* SLIP, then the bug would be in [unpackOSCstream]. check the obvious, e.g. whether [slipenc]/[slipdec] are correctly instantiated in the *packOSCstream abstractions.
in order to find the real cause of your problem, it would be great if you could send the output of [tcpserver] when sending a single simple OSC-message (like "/foo") from liblo.
gmdsart IOhannes
On Fre, 2012-11-09 at 17:19 +0100, IOhannes m zmölnig wrote:
On 11/09/2012 02:51 PM, Jamie Bullock wrote:
So, it seems like [packOSCstream] and [unpackOSCstream] are (un)packing message in a format that can't be used with external clients/servers. Is that correct?
from your experience with liblo, you seem to conclude that: a) all "external clients/servers" are build on top of liblo. b) liblo handles OSC over TCP/IP correctly c) [(un)packOSCstream] handles it wrongly.
anyhow: originally OSC did not define how to transmit OSC-packets over a stream-based protocol like TCP/IP (that has no notion of ending packets). implementations that wanted to transmit OSC over stream-based protocols had to find solutions for themselves.
a few years ago, the OSC specs have been extended, to explicitely mention "SLIP" encoding as the means to packetize streams of OSC-data.
This was added to OSC 1.1, iirc.
OSC 1.0 proposed to prepend to each OSC packet a 4 byte header that contains the byte count of following OSC packet. Old versions of [packOSCstream]/[unpackOSCstream] implemented this proposal.
The 4-byte-header has the big disadvantage of once being out-of-sync, it never catches up again. One would have to re-establish the connection from scratch
[packOSCstream] implements this standard by use of [slipenc].
Exactly, in order to comply with OSC 1.1 (and while breaking compatibility with OSC 1.0)
@Jamie I don't know liblo so well, but you might want to check if it is using the 4-byte-header for OSC-over-TCP.
Roman
On 2012-11-10 05:24, Roman Haefeli wrote:
On Fre, 2012-11-09 at 17:19 +0100, IOhannes m zmölnig wrote:
On 11/09/2012 02:51 PM, Jamie Bullock wrote:
So, it seems like [packOSCstream] and [unpackOSCstream] are (un)packing message in a format that can't be used with external clients/servers. Is that correct?
from your experience with liblo, you seem to conclude that: a) all "external clients/servers" are build on top of liblo. b) liblo handles OSC over TCP/IP correctly c) [(un)packOSCstream] handles it wrongly.
anyhow: originally OSC did not define how to transmit OSC-packets over a stream-based protocol like TCP/IP (that has no notion of ending packets). implementations that wanted to transmit OSC over stream-based protocols had to find solutions for themselves.
a few years ago, the OSC specs have been extended, to explicitely mention "SLIP" encoding as the means to packetize streams of OSC-data.
This was added to OSC 1.1, iirc.
OSC 1.0 proposed to prepend to each OSC packet a 4 byte header that contains the byte count of following OSC packet. Old versions of [packOSCstream]/[unpackOSCstream] implemented this proposal.
The latest version of liblo does exactly this. See liblo-0.26/src/send.c at line 411. It actually sends a packet containing only the length of the OSC message that is sent next.
Martin
On 10 Nov 2012, at 17:10, Martin Peach martin.peach@sympatico.ca wrote:
On 2012-11-10 05:24, Roman Haefeli wrote:
On Fre, 2012-11-09 at 17:19 +0100, IOhannes m zmölnig wrote:
On 11/09/2012 02:51 PM, Jamie Bullock wrote:
So, it seems like [packOSCstream] and [unpackOSCstream] are (un)packing message in a format that can't be used with external clients/servers. Is that correct?
anyhow: originally OSC did not define how to transmit OSC-packets over a stream-based protocol like TCP/IP (that has no notion of ending packets). implementations that wanted to transmit OSC over stream-based protocols had to find solutions for themselves.
a few years ago, the OSC specs have been extended, to explicitely mention "SLIP" encoding as the means to packetize streams of OSC-data.
This was added to OSC 1.1, iirc.
OSC 1.0 proposed to prepend to each OSC packet a 4 byte header that contains the byte count of following OSC packet. Old versions of [packOSCstream]/[unpackOSCstream] implemented this proposal.
The latest version of liblo does exactly this. See liblo-0.26/src/send.c at line 411. It actually sends a packet containing only the length of the OSC message that is sent next.
Thanks everyone, for the helpful suggestions and insights.
It turns out that liblo-0.26 does indeed use the 4-byte header "extension", however the current liblo git revision also contains functionality for using SLIP instead. I tested this with [unpackOSCstream], and I am now able to correctly receive OSC messages over TCP from liblo just fine.
In case anyone finds this thread, looking at a similar issue, the solution on the liblo side is (with liblo from git), to add a call to lo_address_set_flags() on your lo_address before calling lo_send_*, i.e.:
lo_address_set_flags(address, LO_SLIP);
best,
Jamie