Hi:
I am currently developing a system using PD will record users' responses to audio and video files. Essentially, the subject (this is for psychology experiments) will turn a knob indicating their like/dislike (or whatnot) of an audio or video file. For input, I am trying to use a Griffin PowerMate knob with the HID object ( http://at.or.at/hans/pd/hid.html ). I have written a simple external which keeps track of timing (using a timer from flext), and writes the data to a text file.
Timing is important to the researcher. She has a system called the 'black box toolkit' http://www.blackboxtoolkit.co.uk/ and I have done some tests to determine the latency and timing accuracy. For the test, I have a puredata playing an audio file consisting of noise bursts at regular intervals, while I periodically press the knob. The 'Black Box' listens to the sound output and is also wired into the knob. I expected to see the values recorded by my PD external to be offset by 50ms (the latency setting) in comparison to the Black Box recorded values, but I was a bit surprised to see the offset vary between 48 and 62 ms (especially the 48ms!), which gives an uncertainty of 14ms. I was hoping for less than 10ms. The latency is not a problem at all, I am more concerned with accuracy and consistency (so that I can simply compensate for the latency)
Where I should begin to try to improve this uncertainty (would I call it jitter?). What are the likely and most significant factors? Currently I have only tried the test under OS X, with the powermate knob. I will try again under linux, when I can get a LiveCD that will install easily on the old eMacs present in the lab. I will also try with a midi controller, when I find one that I don't mind soldering some test leads onto.
Is the timing error most likely in PD, the HID object, the knob hardware (I know, probably), or just usb? Would a midi controller knob be expected to perform better? And really, what would be the limit of this kind of timing accuracy? Is 10ms as good as I can hope for?
Thanks a million.
-martin
rubber glove wrote:
Hi:
I am currently developing a system using PD will record users' responses to audio and video files. Essentially, the subject (this is for psychology experiments) will turn a knob indicating their like/dislike (or whatnot) of an audio or video file. For input, I am trying to use a Griffin PowerMate knob with the HID object ( http://at.or.at/hans/pd/hid.html ). I have written a simple external which keeps track of timing (using a timer from flext), and writes the data to a text file.
Timing is important to the researcher. She has a system called the 'black box toolkit' http://www.blackboxtoolkit.co.uk/ and I have done some tests to determine the latency and timing accuracy. For the test, I have a puredata playing an audio file consisting of noise bursts at regular intervals, while I periodically press the knob. The 'Black Box' listens to the sound output and is also wired into the knob. I expected to see the values recorded by my PD external to be offset by 50ms (the latency setting) in comparison to the Black Box recorded values, but I was a bit surprised to see the offset vary between 48 and 62 ms (especially the 48ms!), which gives an uncertainty of 14ms. I was hoping for less than 10ms. The latency is not a problem at all, I am more concerned with accuracy and consistency (so that I can simply compensate for the latency)
Where I should begin to try to improve this uncertainty (would I call it jitter?). What are the likely and most significant factors? Currently I have only tried the test under OS X, with the powermate knob. I will try again under linux, when I can get a LiveCD that will install easily on the old eMacs present in the lab. I will also try with a midi controller, when I find one that I don't mind soldering some test leads onto.
Is the timing error most likely in PD, the HID object, the knob hardware (I know, probably), or just usb? Would a midi controller knob be expected to perform better? And really, what would be the limit of this kind of timing accuracy? Is 10ms as good as I can hope for?
The main factor seems to be that pd only processes control messages in between audio frames. Reducing the audio latency seems to bring the jitter down to about 5ms but no further when I try it on WinXP or Linux. The processor speed is not a factor, since I get roughly the same jitter with a 200MHz processor as a 2GHz. Tracking it on a scope using comport or midi objects, the jitter seems to be bimodal: there is a jitter of about 1ms superimposed on a 5ms frame. If the GUI is doing a lot, things get bumped to the next frame, also if you're moving the mouse or running other programs at the same time the OS will steal time for those processes too. I would expect USB to add more jitter, as it passes through a more complex path than serial or MIDI data before being handed to pd. If you need better timing, why not record the audio into one channel along with the button on the other channel? There is essentially no jitter with the audio stream because it passes through the sound hardware at a fixed rate. The button could just emit a click when pressed. Then you have no jitter beyond the time it takes to push the button.
Martin
The main factor seems to be that pd only processes control messages in between audio frames. Reducing the audio latency seems to bring the jitter down to about 5ms but no further when I try it on WinXP or Linux.
Aha... that makes sense to me. So it is not so much just the 'hardware to pd' step, but also the control message passing within PD that is throwing off the timing here.
I would expect USB to add more jitter, as it passes through a more complex path than serial or MIDI data before being handed to pd.
I think I am basically stuck with USB no matter how I go, as the computers involved here are all eMacs. Using midi or serial would involve a USB interface anyway.
If you need better timing, why not record the audio into one channel along with the button on the other channel? There is essentially no jitter with the audio stream because it passes through the sound hardware at a fixed rate. The button could just emit a click when pressed. Then you have no jitter beyond the time it takes to push the button.
That's an interesting possibility, but I am only really using the button clicking as a simple way to test the timing. In actual use, the input would be a range of values from the rotary encoder of the knob... converting that into audio and back into numerical values seems altogether too complicated (frequency modulation?)
However, it brings to mind other possibilities:
If I modify the HID object to convert it into a signal-based external, then pass the values as signals, the timing should be much tighter? This seems to have the advantage that the input values will always be automatically lined up with the audio, no matter what the latency is set to.
Alternatively, If I incorporate the HID code into my external, and process the knob events myself, I would avoid any additional timing errors and be left with only the jitter inherent to the device, no?
Thanks much for the info...
-Martin.
On Jul 4, 2006, at 10:08 AM, rubber glove wrote:
The main factor seems to be that pd only processes control messages in between audio frames. Reducing the audio latency seems to bring the jitter down to about 5ms but no further when I try it on WinXP or
Linux.Aha... that makes sense to me. So it is not so much just the
'hardware to pd' step, but also the control message passing within
PD that is throwing off the timing here.I would expect USB to add more jitter, as it passes through a more complex path than serial or MIDI data before being handed to pd.
I think I am basically stuck with USB no matter how I go, as the
computers involved here are all eMacs. Using midi or serial would
involve a USB interface anyway.If you need better timing, why not record the audio into one channel along with the button on the other channel? There is essentially no jitter with the audio stream because it passes through the sound hardware at a fixed rate. The button could just emit a click when pressed. Then you have no jitter beyond the time it takes to push the button.
That's an interesting possibility, but I am only really using the
button clicking as a simple way to test the timing. In actual use,
the input would be a range of values from the rotary encoder of the
knob... converting that into audio and back into numerical values
seems altogether too complicated (frequency modulation?)However, it brings to mind other possibilities:
If I modify the HID object to convert it into a signal-based
external, then pass the values as signals, the timing should be
much tighter? This seems to have the advantage that the input
values will always be automatically lined up with the audio, no
matter what the latency is set to.Alternatively, If I incorporate the HID code into my external, and
process the knob events myself, I would avoid any additional timing
errors and be left with only the jitter inherent to the device, no?Thanks much for the info...
I would also like to see the [hid] code be as low jitter as
possible. I haven't had a chance to measure it yet tho. Unless
things are really taxing the CPU, I don't think that the Pd messaging
will cause more than ~1.5ms of jitter (1 block of 64 samples divided
by 44100 samples). Perhaps running Pd at 96KHz or 192KHz would
reduce that jitter, provided that the block size remains 64.
I think that the first step is to devise some tests to find the true
source of the jitter. I did some quick searches about USB HID jitter
and didn't find much, except this:
http://elists.resynthesize.com/max-msp/2005/08/1515529/
It seems that this person thinks that Mac OS X's USB MIDI driver has
lower jitter than the USB HID driver. Ultimately, if we find the
source of the problem, I would like to include it in the next version
of [hid].
.hc
Using ReBirth is like trying to play an 808 with a long stick. - David Zicarelli
I managed to run some tests on the HID timing.
The contestants were: -regular HID object in OS X -regular HID object in OS X with blocksize=64 -dissected HID object in OS X (i basically tore the read functions out of the hid code, and put them in my own object, which was recording the values. I did a messy, quick job of it, but...) -ctlin midi object in os x -regular HID object in Linux
the winner, with < 4 ms of timing error was:
linux... I was short on time in the lab (getting the eMac machines booting off the livecd took a bit of time), so I wasn't able to try out different latency settings, or try the midi controller in linux.
the others were all approximately the same, at around or just above a 10 ms spread in timing offsets (I don't have the numbers in front of me.... but One interesting thing is the standard deviation. it seems like mostly the timing error is very close to a mean value, then periodically, there will be a value that is way off, which throws off the average. I haven't made enough measurements to see if there is some sort of periodicity to those 'way off' readings, maybe due to some process switching/memory management by the kernel?)
the smaller blocksize did slightly better than the default setting, the dissected object did slightly better than that... so I would hazard the guess that the delays due to control message passing are not a huge contribution.
and for some reason, the midi object did even worse than the hid object. not at all sure why that is, but it could have to do with the midi devices I was using - who knows.
One thing I realize I should have done is try pd in os x as superuser, as this may give the process a higher priority? I ran sudo pd in linux to have access to the hid devices...
I'm not sure how much more experimentation I will be doing on this, as they are pretty satisfied with < 5 ms error and probably won't pay me to just play around with their expensive timing test equipment....
-martin
On 7/5/06, Hans-Christoph Steiner hans@eds.org wrote:
On Jul 4, 2006, at 10:08 AM, rubber glove wrote:
The main factor seems to be that pd only processes control messages in
between audio frames. Reducing the audio latency seems to bring the jitter down to about 5ms but no further when I try it on WinXP or Linux.
Aha... that makes sense to me. So it is not so much just the 'hardware to pd' step, but also the control message passing within PD that is throwing off the timing here.
I would expect USB to add more jitter, as it passes through a more complex path than serial or MIDI data before being handed to pd.
I think I am basically stuck with USB no matter how I go, as the computers involved here are all eMacs. Using midi or serial would involve a USB interface anyway.
If you need better timing, why not record the audio into one channel along with the button on the other channel? There is essentially no jitter with the audio stream because it passes through the sound hardware at a fixed rate. The button could just emit a click when pressed. Then you have no jitter beyond the time it takes to push the button.
That's an interesting possibility, but I am only really using the button clicking as a simple way to test the timing. In actual use, the input would be a range of values from the rotary encoder of the knob... converting that into audio and back into numerical values seems altogether too complicated (frequency modulation?)
However, it brings to mind other possibilities:
If I modify the HID object to convert it into a signal-based external, then pass the values as signals, the timing should be much tighter? This seems to have the advantage that the input values will always be automatically lined up with the audio, no matter what the latency is set to.
Alternatively, If I incorporate the HID code into my external, and process the knob events myself, I would avoid any additional timing errors and be left with only the jitter inherent to the device, no?
Thanks much for the info...
I would also like to see the [hid] code be as low jitter as possible. I haven't had a chance to measure it yet tho. Unless things are really taxing the CPU, I don't think that the Pd messaging will cause more than ~1.5ms of jitter (1 block of 64 samples divided by 44100 samples). Perhaps running Pd at 96KHz or 192KHz would reduce that jitter, provided that the block size remains 64.
I think that the first step is to devise some tests to find the true source of the jitter. I did some quick searches about USB HID jitter and didn't find much, except this:
http://elists.resynthesize.com/max-msp/2005/08/1515529/
It seems that this person thinks that Mac OS X's USB MIDI driver has lower jitter than the USB HID driver. Ultimately, if we find the source of the problem, I would like to include it in the next version of [hid].
.hc
Using ReBirth is like trying to play an 808 with a long stick. -David Zicarelli
Very interesting, I'd love to see more info, like the numbers and the
patch you used to run these tests. Its a good start.
.hc
On Jul 19, 2006, at 10:35 AM, rubber glove wrote:
I managed to run some tests on the HID timing.
The contestants were: -regular HID object in OS X -regular HID object in OS X with blocksize=64 -dissected HID object in OS X (i basically tore the read functions out of the hid code, and
put them in my own object, which was recording the values. I did a
messy, quick job of it, but...) -ctlin midi object in os x -regular HID object in Linuxthe winner, with < 4 ms of timing error was:
linux... I was short on time in the lab (getting the eMac machines
booting off the livecd took a bit of time), so I wasn't able to try
out different latency settings, or try the midi controller in linux.the others were all approximately the same, at around or just above
a 10 ms spread in timing offsets (I don't have the numbers in front
of me.... but One interesting thing is the standard deviation. it
seems like mostly the timing error is very close to a mean value,
then periodically, there will be a value that is way off, which
throws off the average. I haven't made enough measurements to see
if there is some sort of periodicity to those 'way off' readings,
maybe due to some process switching/memory management by the kernel?)the smaller blocksize did slightly better than the default setting,
the dissected object did slightly better than that... so I would hazard the guess that the delays due to control message
passing are not a huge contribution.and for some reason, the midi object did even worse than the hid
object. not at all sure why that is, but it could have to do with
the midi devices I was using - who knows.One thing I realize I should have done is try pd in os x as
superuser, as this may give the process a higher priority? I ran
sudo pd in linux to have access to the hid devices...I'm not sure how much more experimentation I will be doing on this,
as they are pretty satisfied with < 5 ms error and probably won't
pay me to just play around with their expensive timing test
equipment....-martin
On 7/5/06, Hans-Christoph Steiner hans@eds.org wrote:
On Jul 4, 2006, at 10:08 AM, rubber glove wrote:
The main factor seems to be that pd only processes control
messages in between audio frames. Reducing the audio latency seems to bring the jitter down to about 5ms but no further when I try it on WinXP or
Linux.Aha... that makes sense to me. So it is not so much just the
'hardware to pd' step, but also the control message passing within
PD that is throwing off the timing here.I would expect USB to add more jitter, as it passes through a more complex path than serial or MIDI data before being handed to pd.
I think I am basically stuck with USB no matter how I go, as the
computers involved here are all eMacs. Using midi or serial would
involve a USB interface anyway.If you need better timing, why not record the audio into one channel along with the button on the other channel? There is essentially no jitter with the audio stream because it passes through the sound hardware at a fixed rate. The button could just emit a click when pressed. Then you have no jitter beyond the time it takes to push the button.
That's an interesting possibility, but I am only really using the
button clicking as a simple way to test the timing. In actual use,
the input would be a range of values from the rotary encoder of
the knob... converting that into audio and back into numerical
values seems altogether too complicated (frequency modulation?)However, it brings to mind other possibilities:
If I modify the HID object to convert it into a signal-based
external, then pass the values as signals, the timing should be
much tighter? This seems to have the advantage that the input
values will always be automatically lined up with the audio, no
matter what the latency is set to.Alternatively, If I incorporate the HID code into my external, and
process the knob events myself, I would avoid any additional
timing errors and be left with only the jitter inherent to the
device, no?Thanks much for the info...
I would also like to see the [hid] code be as low jitter as
possible. I haven't had a chance to measure it yet tho. Unless
things are really taxing the CPU, I don't think that the Pd
messaging will cause more than ~1.5ms of jitter (1 block of 64
samples divided by 44100 samples). Perhaps running Pd at 96KHz or
192KHz would reduce that jitter, provided that the block size
remains 64.I think that the first step is to devise some tests to find the
true source of the jitter. I did some quick searches about USB HID
jitter and didn't find much, except this:http://elists.resynthesize.com/max-msp/2005/08/1515529/
It seems that this person thinks that Mac OS X's USB MIDI driver
has lower jitter than the USB HID driver. Ultimately, if we find
the source of the problem, I would like to include it in the next
version of [hid]..hc
--
Using ReBirth is like trying to play an 808 with a long stick. - David Zicarelli
If you are not part of the solution, you are part of the
problem. - Eldridge Cleaver
On 6/30/06, Martin Peach martin.peach@sympatico.ca wrote:
The main factor seems to be that pd only processes control messages in between audio frames.
Do you mean blocks? A few weeks ago someone explicitly said this wasn't true. I don't remember who. He said control messages go when they go, unaffected by audio frames, the audio just doesn't respond to them except between frames.
-Chuckk
Hallo, Chuckk Hubbard hat gesagt: // Chuckk Hubbard wrote:
On 6/30/06, Martin Peach martin.peach@sympatico.ca wrote:
The main factor seems to be that pd only processes control messages in between audio frames.
Do you mean blocks? A few weeks ago someone explicitly said this wasn't true. I don't remember who. He said control messages go when they go, unaffected by audio frames, the audio just doesn't respond to them except between frames.
It's slightly different as I understand this: Control messages get transported every 64 samples, regardless if you set the blocksize lower using e.g. [block~ 1]: It will still get passed every 64 samples.
However Pd also keeps track of time on a much faster scale using its internal "clock", which runs much faster, I think in usecs. Clocks are used for example in [metro] and thus you can run a [metro] with a period of 1msec, even though messages only pass in intervals of about 1.5msec (64 blocks at 44.1kHz). [vline~] is the audio object that can correctly start according to the Clock, if the incoming message is derived from the Clock.
Now I suppose that [hid] does not synchronize to this Clock and thus it is bound to the 64-blocks quantization. I don't know how to "convert" the real time events coming from the HID input to the Clock on Pd, though. Maybe studying m_sched.c very hard can help here.
Ciao
Frank Barknecht _ ______footils.org_ __goto10.org__
-
Chuckk Hubbard -
Frank Barknecht -
Hans-Christoph Steiner -
Martin Peach -
rubber glove