liblo-devel Mailing List for LibLO

Hello Stephen,

I saw the new commits on the repository, thank you very much.


On 22 nov. 2011, at 19:31, Stephen Sinclair wrote:

> I agree that TCP is a great feature.

As of today, 100% of OSC enabled applications built on wireless devices are using UDP as their primitive transport protocol.  This is convenient for the developer, but gives users tools that only seem to work.  I would love to see this change.

Using TCP as the OSC transport protocol would help getting more reliable transport over those wireless networks.  Even though TCP might not be the best solution, supporting it in liblo and other frameworks would at least enable developers to start considering this as an option.  (I have a simple Objective-C example using VVOSC, Bonjour and TCP, those who are interested please ask.)


> Would love to discuss if anyone has experience using OSC over TCP, as I haven't actually used it all
> that much.

I can't say I have much experience, but I made some experiments.  So, I have a more or less working build of OSCulator that allows the user to switch between TCP and UDP, and route messages to TCP or UDP destinations, based on their URL.  I can share this version with you if you are interested and have a Mac.

There are two problems I would love to fix before I can test it more extensively and make a shippable product:

1.  lo_address does not play nicely with socket binding and closing.
    I described this in a post to liblo-devel a short while ago: http://sourceforge.net/mailarchive/message.php?msg_id=28155415.  The solution is not yet clear to my mind.  One approach would be to periodically check that the socket is still valid, maybe by having an API that checks if the sockets are still open ; something that you could integrate in your program's run-loop...

2.  Inter-application messaging using TCP on Mac OS X seems to behave differently from UDP.
    When I use a Wiimote to send data (around 30 kB/s, ie 100 bundles of approximately 300 byte per second), I can see delays and bursts of data that I can't explain (certainly because my limited knowledge in low-level networking).  I wonder if this has to do with the TCP packet window size, and if this can eventually be tuned for OSC.
    It is also possible that the choice of the protocol belongs to the user, based on the application.  Using TCP for inter-application messaging with a Wiimote does not make a lot of sense anyway.


> I would like to know how it compares to other implementations.  I have
> to look more into it but I think there are some further suggestions
> for how to package OSC over a stream connection in the OSC 1.1
> specification.  And there is currently extensive discussion of OSC 2.0
> happening on the OSC-dev list, but I haven't been following it 100%,
> however my impression is that there are suggestions for certain
> transport-level standards.

Here is a discussion from the OSC_dev mailing list circa 2009.  Jeff Koftinoff shares his experience on OSC packet framing: http://lists.create.ucsb.edu/pipermail/osc_dev/2009-August/001735.html


> I know that CNMAT has been proposing to always use SLIP encoding to
> packetize streams, but this is not what liblo currently does.  Instead
> it takes the length-then-body approach.  I happen to think that over a
> reliable connection the latter is better, since it makes memory
> handling a little easier, but I'd like to maintain compatibility with
> other libraries.  Does LibLo perhaps need an API to set TCP stream
> options such as which packetizing protocol to use?

My suggestion would be to support only one framing method at the moment, there is no need to be exhaustive as adoption of OSC over TCP is still relatively rare.

I find Jeff's arguments very strong towards the length-then-body framing style.  I tried it in some of my experiments, and my conclusion is that I did not find necessary to use a SLIP framing method when using OSC over TCP.  Not only I find this approach duplicating some features of TCP, but also error prone since some bytes will have to be properly encoded in order to use SLIP.  Let's keep SLIP for serial line transport.


All the best,
Cam

On Tue, Nov 22, 2011 at 8:52 AM, Camille Troillard
<ca...@os...> wrote:
> I think having IPv6 and TCP support in liblo is a real plus.

I agree that TCP is a great feature.  Would love to discuss if anyone
has experience using OSC over TCP, as I haven't actually used it all
that much.

I would like to know how it compares to other implementations.  I have
to look more into it but I think there are some further suggestions
for how to package OSC over a stream connection in the OSC 1.1
specification.  And there is currently extensive discussion of OSC 2.0
happening on the OSC-dev list, but I haven't been following it 100%,
however my impression is that there are suggestions for certain
transport-level standards.

I know that CNMAT has been proposing to always use SLIP encoding to
packetize streams, but this is not what liblo currently does.  Instead
it takes the length-then-body approach.  I happen to think that over a
reliable connection the latter is better, since it makes memory
handling a little easier, but I'd like to maintain compatibility with
other libraries.  Does LibLo perhaps need an API to set TCP stream
options such as which packetizing protocol to use?

Steve

Hi Stephen,


On 21 nov. 2011, at 20:09, Stephen Sinclair wrote:

> After a month of working on an unrelated project I finally have time
> to get back to this :)
> Sorry for the delay.

Thank you!
I appreciate you spend time on this.


> I've just been testing this IPV6 patch on Linux (Ubuntu 10.10) and
> iMac (10.6.8), and can report the following.  With PF_UNSPEC  and
> V6ONLY enabled on the server, if the client specifies the hostname as:
> 
> Ubuntu10.10      msg received     lo_send returned
> "localhost"      yes                     number of bytes
> "::1"                 yes                     number of bytes
> "127.0.0.1"      no                       number of bytes
> "::FFFF:127.0.0.1"  no               number of bytes
> 
> OSX 10.6.8
> "localhost"      no                       number of bytes
> "::1"                 yes                     number of bytes
> "127.0.0.1"      no                       number of bytes
> "::FFFF:127.0.0.1"  no               number of bytes
> 
> With IPV6ONLY disabled,
> 
> Ubuntu 10.10:
> "localhost"      yes                     number of bytes
> "::1"                 yes                     number of bytes
> "127.0.0.1"      yes                     number of bytes
> "::FFFF:127.0.0.1"  yes             number of bytes
> 
> OSX 10.6.8:
> "localhost"      yes                     number of bytes
> "::1"                 yes                     number of bytes
> "127.0.0.1"      yes                     number of bytes
> "::FFFF:127.0.0.1"  yes             number of bytes
> 
> So, I never observed lo_send() returning error 22, contrary to your
> results, which is strange.  It could simply be a difference between
> the order of the NICs in the two machines, the wifi vs. wired
> configuration, etc.  (I was testing on a wired ethernet connection.)

Right, I was testing using the wifi adapter.


> The only difference with Ubuntu seems to be that the result of looking
> up "localhost" results in IPv4 on Mac OS X but in IPv6 on Linux.

I prefer the results of Ubuntu.  But if you have a look at /etc/hosts on Mac OS X it does indeed have the IPv6 addresses for localhost.  Oh well ...


> However, it's clear that V6ONLY is working on both operating systems,
> and using PF_UNSPEC is correct when "::FFFF" is prepended.  Moreover I
> never observed a case where the loop over addrinfo actually ran more
> than once.  But I see no problem in iterating and saving the results,
> if that potentially solves a problem.  It's easy to simply save the
> result in lo_address if the loop discovers a successful addrinfo later
> in the list, so it should only execute the loop once.

Sounds good to me!


> I think I can safely commit this patch (done) but some independent
> testing would be appreciated.

Thank you Steve.
I'll be back in a short while with some questions about liblo and TCP.

I think having IPv6 and TCP support in liblo is a real plus.


Best,
Cam

Hi,

After a month of working on an unrelated project I finally have time
to get back to this :)
Sorry for the delay.

I've just been testing this IPV6 patch on Linux (Ubuntu 10.10) and
iMac (10.6.8), and can report the following.  With PF_UNSPEC  and
V6ONLY enabled on the server, if the client specifies the hostname as:

Ubuntu10.10      msg received     lo_send returned
"localhost"      yes                     number of bytes
"::1"                 yes                     number of bytes
"127.0.0.1"      no                       number of bytes
"::FFFF:127.0.0.1"  no               number of bytes

OSX 10.6.8
"localhost"      no                       number of bytes
"::1"                 yes                     number of bytes
"127.0.0.1"      no                       number of bytes
"::FFFF:127.0.0.1"  no               number of bytes

With IPV6ONLY disabled,

Ubuntu 10.10:
"localhost"      yes                     number of bytes
"::1"                 yes                     number of bytes
"127.0.0.1"      yes                     number of bytes
"::FFFF:127.0.0.1"  yes             number of bytes

OSX 10.6.8:
"localhost"      yes                     number of bytes
"::1"                 yes                     number of bytes
"127.0.0.1"      yes                     number of bytes
"::FFFF:127.0.0.1"  yes             number of bytes

So, I never observed lo_send() returning error 22, contrary to your
results, which is strange.  It could simply be a difference between
the order of the NICs in the two machines, the wifi vs. wired
configuration, etc.  (I was testing on a wired ethernet connection.)
The only difference with Ubuntu seems to be that the result of looking
up "localhost" results in IPv4 on Mac OS X but in IPv6 on Linux.

However, it's clear that V6ONLY is working on both operating systems,
and using PF_UNSPEC is correct when "::FFFF" is prepended.  Moreover I
never observed a case where the loop over addrinfo actually ran more
than once.  But I see no problem in iterating and saving the results,
if that potentially solves a problem.  It's easy to simply save the
result in lo_address if the loop discovers a successful addrinfo later
in the list, so it should only execute the loop once.

I think I can safely commit this patch (done) but some independent
testing would be appreciated.

thanks,
Steve


On Mon, Oct 3, 2011 at 3:49 AM, Camille Troillard <ca...@os...> wrote:
>
> On 3 oct. 2011, at 05:53, Stephen Sinclair wrote:
>
>> Since my last email I have been trying to figure out whether this
>> iteration is truly needed, or whether we can determine upfront which
>> of the returned addresses matches best.  In general I think the
>> connectionless nature of UDP means that checking the return code of
>> sendto() as the basis for iteration over addrinfo is problematic.
>>
>> I have now been doing similar tests, so I thought I should try to
>> write down the results, as it might make a nice reference sheet.  I
>> have been playing with the specified families of the server and
>> client, and how it works with the hostname format.  These tests were
>> done on my EEEPC running Ubuntu 10.10.
>>
>> With ENABLE_IPV6, and PF_UNSPEC as the hint for getaddrinfo at client,
>> if the server is initialized with PF_INET (IPv4):
>>
>> - client sends to "127.0.0.1"
>>  getaddrinfo returns 1 address, family AF_INET
>>  message is received as usual
>>
>> - client sends to "::FFFF:127.0.0.1"
>>  getaddrinfo returns 1 address, family AF_INET6
>>  message is received
>>
>> - client sends to "::1"
>>  getaddrinfo returns 1 address, family AF_INET6
>>  message is NOT received, expected
>>  sendto() does not return an error
>>
>> - client sends to "localhost"
>>  getaddrinfo returns 2 addresses, family AF_INET6 and AF_INET respectively
>>  message is NOT received on first address, second address is NOT
>> TRIED because sendto() does not return error
>>  message IS received if I force it to use the second address (AF_INET)
>>
>> Similarly with PF_UNSPEC at client, if the server is initialized with
>> PF_INET6 (IPv6), and with IPV6_V6ONLY explicitly turned ON.
>>
>> - client sends to "127.0.0.1"
>>  getaddrinfo returns 1 address, family AF_INET
>>  message is NOT received, expected
>>  sendto() does not return an error
>>
>> - client sends to "::FFFF:127.0.0.1"
>>  getaddrinfo returns 1 address, family AF_INET6
>>  message is NOT received, expected
>>  sendto() does not return an error
>>
>> - client sends to "::1"
>>  getaddrinfo returns 1 address, family AF_INET6
>>  message is received
>>
>> - client sends to "localhost"
>>  getaddrinfo returns 2 addresses, family AF_INET6 and AF_INET respectively
>>  message is received on first address
>>  message is NOT received if I force it to use the second address
>> (AF_INET), expected
>>
>> Finally with PF_UNSPEC at client, if the server is initialized with
>> PF_INET6 (IPv6), and with IPV6_V6ONLY explicitly turned OFF.
>>
>> - client sends to "127.0.0.1"
>>  getaddrinfo returns 1 address, family AF_INET
>>  message IS received
>>
>> - client sends to "::FFFF:127.0.0.1"
>>  getaddrinfo returns 1 address, family AF_INET6
>>  message IS received
>>
>> - client sends to "::1"
>>  getaddrinfo returns 1 address, family AF_INET6
>>  message is received
>>
>> - client sends to "localhost"
>>  getaddrinfo returns 2 addresses, family AF_INET6 and AF_INET respectively
>>  message is received on first address
>>  message IS ALSO received if I force it to use the second address (AF_INET)
>>
>> So, I think this makes the behaviour of IPV6_V6ONLY pretty clear.
>> However, it also indicates that checking the return code of sendto()
>> is not the right way to choose which address to use if more than one
>> is available.  In fact, to select, it seems to be really necessary to
>> explicitly know whether the receiver is IPv4 or IPv6, then if two
>> addresses are returned the correct one can be chosen.  But I'm not
>> sure what to do in the case where this is unknown by the client.  The
>> obvious thing is to disable IPV6_V6ONLY, in which case it should "just
>> work", but this means the client depends on the server having this
>> flag disabled.
>
> That is a very good point.
>
> However, I just tested, and the behavior on Mac OS X 10.6.8 seems a bit different.
> With PF_UNSPEC at the client, and IPV6_V6ONLY disabled on the server, without looping over the results of getaddrinfo, sendto returns an error 22.
>
> So in my opinion, iterating is still necessary.
>
> ----
> I just discovered one very curious and unexpected behavior:
>
> I can launch one server with IPV6_V6ONLY on port XYZW, and an IPV4 only server on the same port.  I was not expecting this!  The result is that every client request configured with explicit IPv4 address are sent to the IPv4 server, and all other (including osc://localhost:XYZW) are sent to the other server.
>
> When I stopped my IPv6 server, all the requests were then sent to the IPv4 server, even if the address was specified as a IPv6 address.
> ----
>
>
>> It's possible to specify IPv4 or IPv6 by the structure of the
>> numerical IP string, but in the case where a hostname is given I'm not
>> sure how to specify "connect to this host by IPv6".  Perhaps it's
>> supposed to be up to the name resolution service to decide for you?
>> (Like, if you browse to an IPv6 website is it up to the DNS server to
>> inform the system to connect via IPv6?)
>
> Yes, in my opinion this is the job of the DNS service.
>
>
>> Anyways, maybe a good-enough solution is just to always select IPv6 if
>> it is available, and maybe let this be overridden at runtime if IPv4
>> needs to be selected.  (e.g. lo_address_enable_ipv6(boolean))
>
> Since sento does not return an error on Linux, this would be a good solution, but on Mac OS X this option would not be needed if we loop over the results of getaddrinfo.  Do you think both solutions (lo_address_enable_ipv6 and iteration) work together?
>
>
>> Also, I notice the sscanf incantations in the URL parsing code may not
>> be quite up to the job of accepting URLs like "osc.udp://[::1]:9000",
>> so that might need looking at.
>
> I am not sure of this, it looks like the second statement in lo_url_get_hostname correctly takes in account bracket characters.  Do you see another case where this would not work?
>
>
> As a last note, I noticed that the code I added to prepend ::FFFF: to IPv4 dotted addresses when IPv6 is enabled was not taking in account the NUL character.  In address.c at line 386, the correction is:
>
> #ifdef ENABLE_IPV6
>        hints.ai_family = PF_UNSPEC;
>
>        if (is_dotted_ipv4_address(a->host)) {
>                size_t len = strlen(a->host) + 1;
>                host = calloc(len + 7, sizeof(char));
>                strcpy(host, "::FFFF:");
>                strncpy(host + 7, a->host, len);
>        }
> #else
>
>
>
> ---
> Cam
>
>
> ------------------------------------------------------------------------------
> All the data continuously generated in your IT infrastructure contains a
> definitive record of customers, application performance, security
> threats, fraudulent activity and more. Splunk takes this data and makes
> sense of it. Business sense. IT sense. Common sense.
> http://p.sf.net/sfu/splunk-d2dcopy1
> _______________________________________________
> liblo-devel mailing list
> lib...@li...
> https://lists.sourceforge.net/lists/listinfo/liblo-devel
>
>