Hi Gavin!
On 09/12/14 19:27, Gavin wrote:
Is it ok to email you about this or would you prefer to keep it to the forums?
I hope you don't mind me CC'ing this to the Pd-list.
I'm happy you emailed about this because it is exactly what I have been working on lately when I find spare time.
I'm a bit confused as to the network options with pddroidparty. There is netro which seems to be based on netsend and x-netreceive but also droidnetreceive and also netreceive! Which is the recommended or latest one?
The native netsend and netreceive now work in the version of Pd that ships with PdDroidParty. The other libraries are deprecated. Sorry for the confusion - there was a long standing bug with netreceive not working in libpd that is now resolved. x-netreceive was just my way of commenting out netreceive - you can change it back to netreceive and remove the top level droidnetreceive hack object and netro should just work again.
Netro is an abstraction for doing timing synchronisation of Pd patches that runs on top of netreceive. I haven't updated it for a long time because I've been working on something that does the job better:
http://github.com/chr15m/SyncJams/
SyncJams uses OSC underneath and so it can interoperate more widely with other software. There is also a Python implementation and I am hoping for implementations in other languages (especially Supercollider) down the track. Patches welcome!
SyncJams provides a way to not only sync metronomes during live performance but also a sort of distributed hash table ("set-state") where everybody involved in a jam can share state information like controller position, song key, etc. There is also a generic conduit for in-time events ("message") to be propagated in order such as drum beat hits from a live drummer.
I've been testing on Android, and latency aside things are working pretty smoothly now.
Also is there anyway to debug/print to a console on a phone?
If you use Pd's print object you can then use adb logcat to see what
comes out - you'll need to have your phone in debugging mode and
connected via USB or over TCP to adb.
I'm hoping to guesstimate latency by calculating the time it takes to send data from one device to another and back and sync phasors on separate devices.
Have you tried this?
So latency is the exact problem I am tackling at the moment.
After speaking to Peter Brinkmann, the resident Android audio expert, the outlook is as abysmal as ever. The way audio is implemented on Android means you can't ever really know with 100% certainty how long it takes between a software event and the speaker actually emitting the triggered sound.
My approach, which is hopefully "good enough" is similar to what you suggest, except that a single device can send a signal to its own microphone to measure the round trip time. Hopefully half of the round trip time will give the approximate output latency. This is what the android-latency-tester patch did and I will probably use that same method (with the bonk~ object).
http://puredata.info/docs/AndroidLatency/
I tested a FFT based cross correlation approach to detect the same-device latency, but it wasn't really robust enough. That is in the SyncJams repository if you want to take a look. Maybe I did something wrong though as my DSP math is not very strong.
There is also Raph Levien's audio buffer size tool:
https://play.google.com/store/apps/details?id=com.levien.audiobuffersize
Which uploads its results for different devices: http://audiobuffersize.appspot.com/
Code: https://code.google.com/p/high-performance-audio/source/browse/audio-buffer-...
In the end I think the approach on Android is going to be:
- Alert the user the first time they run the patch that latency needs
to be tested.
- "Please turn up your speakers while we run the test."
- Collect the timing data using bonk~ or FFT cross correlation or
whatever and write it to disk.
- Everybody who makes an Android Pd app can use the same filename,
location, and format for the data (/sdcard/Pd/latency-test.txt) so hopefully it only has to be done once.
- Use the values in that file to compensate for latency.
- Provide a menu option for the user to re-run the compensation if they
feel latency is getting out of hand.
PS One more thing - where can I get x-netreceive so I can install it on my laptop. Is it the same as xnetreceive?
x-netreceive is not a real object - it's what I used to comment out the existing netreceive object in netro when that wasn't working under PdDroidParty. What you can do is replace it with netreceive instead, and remove droidnetreceive entirely. Things should work after that.
Let us know how you go!
Cheers,
Chris.
Thanks for your comprehensive reply. Hopefully this conversation should now be on list.
Syncjams looks very interesting I will try it out today! I opened it on my laptop and it seems I need the netsync object. I have found the repository on github - do I need to grab this?
Or is netsync bundled with pddroidparty? Which is the latest version? - is downloading 276 from the website sufficient or so I need to download it from github and compile it or something?
I'm finding it a bit difficult to know if the objects are missing on my phone. Bit of a noob at this. It's like blind coding!
I realised that Android has severe audio latency issues but I thought these were largely when using audio in. I thought my latency was largely created by the network but now I am questioning this. In any case yes your audio approach seems like a fair solution. Hopefully lollipop will improve this and in about 5 years it will be universally adopted!
Right I'll try and get syncjams to work - I'm currently using 1 phone and 1 laptop - if for some reason this will never work and I need to use 2 devices, please someone put me out of my misery!
On Thu, Dec 11, 2014 at 2:08 AM, Chris McCormick chris@mccormick.cx wrote:
Hi Gavin!
On 09/12/14 19:27, Gavin wrote:
Is it ok to email you about this or would you prefer to keep it to the forums?
I hope you don't mind me CC'ing this to the Pd-list.
I'm happy you emailed about this because it is exactly what I have been working on lately when I find spare time.
I'm a bit confused as to the network options with pddroidparty. There is netro which seems to be based on netsend and x-netreceive but also droidnetreceive and also netreceive! Which is the recommended or latest one?
The native netsend and netreceive now work in the version of Pd that ships with PdDroidParty. The other libraries are deprecated. Sorry for the confusion - there was a long standing bug with netreceive not working in libpd that is now resolved. x-netreceive was just my way of commenting out netreceive - you can change it back to netreceive and remove the top level droidnetreceive hack object and netro should just work again.
Netro is an abstraction for doing timing synchronisation of Pd patches that runs on top of netreceive. I haven't updated it for a long time because I've been working on something that does the job better:
http://github.com/chr15m/SyncJams/
SyncJams uses OSC underneath and so it can interoperate more widely with other software. There is also a Python implementation and I am hoping for implementations in other languages (especially Supercollider) down the track. Patches welcome!
SyncJams provides a way to not only sync metronomes during live performance but also a sort of distributed hash table ("set-state") where everybody involved in a jam can share state information like controller position, song key, etc. There is also a generic conduit for in-time events ("message") to be propagated in order such as drum beat hits from a live drummer.
I've been testing on Android, and latency aside things are working pretty smoothly now.
Also is there anyway to debug/print to a console on a phone?
If you use Pd's print object you can then use
adb logcatto see what comes out - you'll need to have your phone in debugging mode and connected via USB or over TCP to adb.I'm hoping to guesstimate latency by calculating the time it takes to send data from one device to another and back and sync phasors on separate devices.
Have you tried this?
So latency is the exact problem I am tackling at the moment.
After speaking to Peter Brinkmann, the resident Android audio expert, the outlook is as abysmal as ever. The way audio is implemented on Android means you can't ever really know with 100% certainty how long it takes between a software event and the speaker actually emitting the triggered sound.
My approach, which is hopefully "good enough" is similar to what you suggest, except that a single device can send a signal to its own microphone to measure the round trip time. Hopefully half of the round trip time will give the approximate output latency. This is what the android-latency-tester patch did and I will probably use that same method (with the bonk~ object).
http://puredata.info/docs/AndroidLatency/
I tested a FFT based cross correlation approach to detect the same-device latency, but it wasn't really robust enough. That is in the SyncJams repository if you want to take a look. Maybe I did something wrong though as my DSP math is not very strong.
There is also Raph Levien's audio buffer size tool:
https://play.google.com/store/apps/details?id=com.levien.audiobuffersize
Which uploads its results for different devices: http://audiobuffersize.appspot.com/
Code:
https://code.google.com/p/high-performance-audio/source/browse/audio-buffer-...
In the end I think the approach on Android is going to be:
- Alert the user the first time they run the patch that latency needs
to be tested.
- "Please turn up your speakers while we run the test."
- Collect the timing data using bonk~ or FFT cross correlation or
whatever and write it to disk.
- Everybody who makes an Android Pd app can use the same filename,
location, and format for the data (/sdcard/Pd/latency-test.txt) so hopefully it only has to be done once.
- Use the values in that file to compensate for latency.
- Provide a menu option for the user to re-run the compensation if they
feel latency is getting out of hand.
PS One more thing - where can I get x-netreceive so I can install it on my laptop. Is it the same as xnetreceive?
x-netreceive is not a real object - it's what I used to comment out the existing netreceive object in netro when that wasn't working under PdDroidParty. What you can do is replace it with netreceive instead, and remove droidnetreceive entirely. Things should work after that.
Let us know how you go!
Cheers,
Chris.
I've got this working now!
In the version I had there is a small bug/typo which prevents the sliders and toggles from updating.
In the netbus.pd file there is an abstraction on the send side, pd oscformat Inside it there is a message box 'set /syncjams$1' I think this needs to read 'set /syncjams/$1' or the route object on the receive side doesn't do what it is supposed too.
Annoyingly I haven't managed to get adb access to my device yet - it is reported as offline and I lost the will to live trying to find out why. But luckily I worked out what I needed to get syncjams to work.
The latency on my device, a Moto G, seems to be around 100ms. The ping time from my laptop to my phone is 3ms so I guess I can pretty much discount that!
On Thu, Dec 11, 2014 at 11:32 AM, Gavin info@digitalfunfair.co.uk wrote:
Thanks for your comprehensive reply. Hopefully this conversation should now be on list.
Syncjams looks very interesting I will try it out today! I opened it on my laptop and it seems I need the netsync object. I have found the repository on github - do I need to grab this?
Or is netsync bundled with pddroidparty? Which is the latest version? - is downloading 276 from the website sufficient or so I need to download it from github and compile it or something?
I'm finding it a bit difficult to know if the objects are missing on my phone. Bit of a noob at this. It's like blind coding!
I realised that Android has severe audio latency issues but I thought these were largely when using audio in. I thought my latency was largely created by the network but now I am questioning this. In any case yes your audio approach seems like a fair solution. Hopefully lollipop will improve this and in about 5 years it will be universally adopted!
Right I'll try and get syncjams to work - I'm currently using 1 phone and 1 laptop - if for some reason this will never work and I need to use 2 devices, please someone put me out of my misery!
On Thu, Dec 11, 2014 at 2:08 AM, Chris McCormick chris@mccormick.cx wrote:
Hi Gavin!
On 09/12/14 19:27, Gavin wrote:
Is it ok to email you about this or would you prefer to keep it to the forums?
I hope you don't mind me CC'ing this to the Pd-list.
I'm happy you emailed about this because it is exactly what I have been working on lately when I find spare time.
I'm a bit confused as to the network options with pddroidparty. There is netro which seems to be based on netsend and x-netreceive but also droidnetreceive and also netreceive! Which is the recommended or latest one?
The native netsend and netreceive now work in the version of Pd that ships with PdDroidParty. The other libraries are deprecated. Sorry for the confusion - there was a long standing bug with netreceive not working in libpd that is now resolved. x-netreceive was just my way of commenting out netreceive - you can change it back to netreceive and remove the top level droidnetreceive hack object and netro should just work again.
Netro is an abstraction for doing timing synchronisation of Pd patches that runs on top of netreceive. I haven't updated it for a long time because I've been working on something that does the job better:
http://github.com/chr15m/SyncJams/
SyncJams uses OSC underneath and so it can interoperate more widely with other software. There is also a Python implementation and I am hoping for implementations in other languages (especially Supercollider) down the track. Patches welcome!
SyncJams provides a way to not only sync metronomes during live performance but also a sort of distributed hash table ("set-state") where everybody involved in a jam can share state information like controller position, song key, etc. There is also a generic conduit for in-time events ("message") to be propagated in order such as drum beat hits from a live drummer.
I've been testing on Android, and latency aside things are working pretty smoothly now.
Also is there anyway to debug/print to a console on a phone?
If you use Pd's print object you can then use
adb logcatto see what comes out - you'll need to have your phone in debugging mode and connected via USB or over TCP to adb.I'm hoping to guesstimate latency by calculating the time it takes to send data from one device to another and back and sync phasors on separate devices.
Have you tried this?
So latency is the exact problem I am tackling at the moment.
After speaking to Peter Brinkmann, the resident Android audio expert, the outlook is as abysmal as ever. The way audio is implemented on Android means you can't ever really know with 100% certainty how long it takes between a software event and the speaker actually emitting the triggered sound.
My approach, which is hopefully "good enough" is similar to what you suggest, except that a single device can send a signal to its own microphone to measure the round trip time. Hopefully half of the round trip time will give the approximate output latency. This is what the android-latency-tester patch did and I will probably use that same method (with the bonk~ object).
http://puredata.info/docs/AndroidLatency/
I tested a FFT based cross correlation approach to detect the same-device latency, but it wasn't really robust enough. That is in the SyncJams repository if you want to take a look. Maybe I did something wrong though as my DSP math is not very strong.
There is also Raph Levien's audio buffer size tool:
https://play.google.com/store/apps/details?id=com.levien.audiobuffersize
Which uploads its results for different devices: http://audiobuffersize.appspot.com/
Code:
https://code.google.com/p/high-performance-audio/source/browse/audio-buffer-...
In the end I think the approach on Android is going to be:
- Alert the user the first time they run the patch that latency needs
to be tested.
- "Please turn up your speakers while we run the test."
- Collect the timing data using bonk~ or FFT cross correlation or
whatever and write it to disk.
- Everybody who makes an Android Pd app can use the same filename,
location, and format for the data (/sdcard/Pd/latency-test.txt) so hopefully it only has to be done once.
- Use the values in that file to compensate for latency.
- Provide a menu option for the user to re-run the compensation if they
feel latency is getting out of hand.
PS One more thing - where can I get x-netreceive so I can install it on my laptop. Is it the same as xnetreceive?
x-netreceive is not a real object - it's what I used to comment out the existing netreceive object in netro when that wasn't working under PdDroidParty. What you can do is replace it with netreceive instead, and remove droidnetreceive entirely. Things should work after that.
Let us know how you go!
Cheers,
Chris.
-- Gavin www.digitalfunfair.co.uk
Hi Gavin,
On 15/12/14 07:07, Gavin wrote:
In the version I had there is a small bug/typo which prevents the sliders and toggles from updating.
In the netbus.pd file there is an abstraction on the send side, pd oscformat Inside it there is a message box 'set /syncjams$1' I think this needs to read 'set /syncjams/$1' or the route object on the receive side doesn't do what it is supposed too.
It's actually not a bug - that is the way it is supposed to be. The bug is that I am not enforcing or adding a leading slash "/" on the variable names used. If you change your sync variable names to valid OSC namespaces - starting with a slash - everything should work without modifying the library, and that is a more robust fix for the future.
I will put something into SyncJams to either add the OSC-compliant leading slash or to warn the user about the missing slash on their variable names.
Annoyingly I haven't managed to get adb access to my device yet - it is reported as offline and I lost the will to live trying to find out why.
One fix for this on some platforms is to run adb as root. I often restart adb as root when I am doing Android development as follows:
adb kill-server
sudo which adb start-server
Note this is a little bit less secure than having correct permissions set.
The latency on my device, a Moto G, seems to be around 100ms. The ping time from my laptop to my phone is 3ms so I guess I can pretty much discount that!
If this is just for your own purposes (i.e. to use that specific device and sync to your laptop) you can manually tune the latency on the phone yourself to account for the 100ms. I'll update the latency compensator patch to allow for this until I get the latency calibration thing working.
For a while I was testing with two Android devices which both happened to have identical 100ms latency and so the sync was very nice.
Thanks for testing the library out - your feedback is invaluable!
Cheers,
Chris.
Quick update - I got rid of the 255.255.255.255 broadcast option and it seemed to make things much better, though still not perfect.
Is the patch meant to choose Android network broadcasting (192.168.43.255) if available and if not then fall back to lan blanket broadcasting? I'm not sure but for me, both spigots were open and I think the network was getting congested.
Now it works much better but there is still some lag and I am getting alot of $2 : argument out of range messages - not sure where they originate from.
Hi Gavin,
My Moto G doesn't report a unique network id for some reason.
This is a fairly critical bug. The protocol won't behave well if a node is lacking a uid. After looking at the uid object I think this could happen if a device lacks the "loopback" interface 127.0.0.1 - so I've changed the code so that a) it tries to gather network entropy by connecting to 0.0.0.0 instead and b) it falls back to just using the startup entropy if the network entropy fails.
I have pushed this code - do you mind testing again?
On 16/12/14 06:48, Gavin wrote:
Quick update - I got rid of the 255.255.255.255 broadcast option and it seemed to make things much better, though still not perfect.
Is the patch meant to choose Android network broadcasting (192.168.43.255) if available and if not then fall back to lan blanket broadcasting? I'm not sure but for me, both spigots were open and I think the network was getting congested.
The protocol is designed so that it can send on as many interfaces as possible redundantly. This is so in future I can drop in other transports like mesh networking and whatever packets make it through first will be the ones that get used. Because each message gets a unique ID the protocol can easily drop redundant messages.
Because of the way that message retries work (based on a simple checksum of the state hash) if something is broken with the uids it will probably result in a lot of traffic as the nodes try and get the broken node up to date.
I probably need to include a retry counter where if a node is continuously behaving badly it gets ignored by the other nodes eventually, say after 100 retries. I'll add that to the TODO.
Now it works much better but there is still some lag and I am getting alot of $2 : argument out of range messages - not sure where they originate from.
That's not good. Those were plaguing me and I could not find the source, but they went away when I got all the other bugs ironed out. Can you let me know if the latest from GitHub exhibits the same symptoms?
https://github.com/chr15m/SyncJams
Thanks for testing!
Cheers,
Chris.
Still the same I'm afraid. No network id displayed on the Moto G although I did get one from the cheap tablet. Pinging 127.0.0.1 in a terminal emulator seems to work on the Moto G in any case.
I couldn't get adb to work - still reporting offline even with sudo! Maybe that would give us a clue as to what is going on?
On Wed, Dec 17, 2014 at 2:53 AM, Chris McCormick chris@mccormick.cx wrote:
Hi Gavin,
My Moto G doesn't report a unique network id for some reason.
This is a fairly critical bug. The protocol won't behave well if a node is lacking a uid. After looking at the uid object I think this could happen if a device lacks the "loopback" interface 127.0.0.1 - so I've changed the code so that a) it tries to gather network entropy by connecting to 0.0.0.0 instead and b) it falls back to just using the startup entropy if the network entropy fails.
I have pushed this code - do you mind testing again?
On 16/12/14 06:48, Gavin wrote:
Quick update - I got rid of the 255.255.255.255 broadcast option and it seemed to make things much better, though still not perfect.
Is the patch meant to choose Android network broadcasting (192.168.43.255) if available and if not then fall back to lan blanket broadcasting? I'm not sure but for me, both spigots were open and I think the network was getting congested.
The protocol is designed so that it can send on as many interfaces as possible redundantly. This is so in future I can drop in other transports like mesh networking and whatever packets make it through first will be the ones that get used. Because each message gets a unique ID the protocol can easily drop redundant messages.
Because of the way that message retries work (based on a simple checksum of the state hash) if something is broken with the uids it will probably result in a lot of traffic as the nodes try and get the broken node up to date.
I probably need to include a retry counter where if a node is continuously behaving badly it gets ignored by the other nodes eventually, say after 100 retries. I'll add that to the TODO.
Now it works much better but there is still some lag and I am getting alot of $2 : argument out of range messages - not sure where they originate from.
That's not good. Those were plaguing me and I could not find the source, but they went away when I got all the other bugs ironed out. Can you let me know if the latest from GitHub exhibits the same symptoms?
https://github.com/chr15m/SyncJams
Thanks for testing!
Cheers,
Chris.
Digging a bit deeper, it seems as if the realtime object is not working on my Moto G
I'll attach a demo patch which should display the realtime output when hitting the trigger.
Works on my laptop - no effect on the Moto G.
Any ideas?
On Wed, Dec 17, 2014 at 2:53 AM, Chris McCormick chris@mccormick.cx wrote:
Hi Gavin,
My Moto G doesn't report a unique network id for some reason.
This is a fairly critical bug. The protocol won't behave well if a node is lacking a uid. After looking at the uid object I think this could happen if a device lacks the "loopback" interface 127.0.0.1 - so I've changed the code so that a) it tries to gather network entropy by connecting to 0.0.0.0 instead and b) it falls back to just using the startup entropy if the network entropy fails.
I have pushed this code - do you mind testing again?
On 16/12/14 06:48, Gavin wrote:
Quick update - I got rid of the 255.255.255.255 broadcast option and it seemed to make things much better, though still not perfect.
Is the patch meant to choose Android network broadcasting (192.168.43.255) if available and if not then fall back to lan blanket broadcasting? I'm not sure but for me, both spigots were open and I think the network was getting congested.
The protocol is designed so that it can send on as many interfaces as possible redundantly. This is so in future I can drop in other transports like mesh networking and whatever packets make it through first will be the ones that get used. Because each message gets a unique ID the protocol can easily drop redundant messages.
Because of the way that message retries work (based on a simple checksum of the state hash) if something is broken with the uids it will probably result in a lot of traffic as the nodes try and get the broken node up to date.
I probably need to include a retry counter where if a node is continuously behaving badly it gets ignored by the other nodes eventually, say after 100 retries. I'll add that to the TODO.
Now it works much better but there is still some lag and I am getting alot of $2 : argument out of range messages - not sure where they originate from.
That's not good. Those were plaguing me and I could not find the source, but they went away when I got all the other bugs ironed out. Can you let me know if the latest from GitHub exhibits the same symptoms?
https://github.com/chr15m/SyncJams
Thanks for testing!
Cheers,
Chris.
-
Chris McCormick -
Gavin