Hans, thanks for your comments
On Mon, Oct 24, 2011 at 7:35 PM, Hans-Christoph Steiner hans@at.or.at wrote:
That looks really good on many levels. :) I like the layout, I hadn't thought of standard deviation, that makes sense as long as we can specify "exact" as a possibility. My guess is that some of this stuff should produce the same bit sequence every time, but I could be wrong there.
If you specify tolerance zero, that's exact. But we found that tolerance is needed for different reasons:
- floats are stored as text by Pd, and differences between computed and stored values occur because of the truncated stored floats - there's small differences between single precision and double precision Pd - objects with a 'memory' (like IIR filters) produce slightly different results if you test them repeatedly with the same input sequence
Perhaps the subpatches should be abstractions as part of a 'test' library. If there was a bug or a new feature needed in any of those subpatches, it would be rough to have to modify all of the tests.
Of course, thanks. I'll redesign it in that sense.
About the sample rate, it should be possible to have each patch set the sample rate it needs. You can see hcs/get-audio-dialog-help.pd or the mediasettings library for ways to do that. I think we'll want to test at different sample rates some day, but if its easier for now, we can stick to 44100.
The idea was to only use vanilla objects around the object under test, so you don't depend on external libs for testing. Do you know if it's possible to send standard messages like 'audio-dialog 1 0 0 0 2 0 0 0 0 0 0 0 2 0 0 0 44100 20 0' or 'audio-dialog 1 0 0 0 2 0 0 0 0 0 0 0 2 0 0 0 48000 20 0' to pd on all systems, without requesting data first? I don't think so. If the device has different number of channels (instead of 2), Pd can not sync. And if the device doesn't allow the alternative samplerate, it won't work either. Samplerate 44100 is safest.
By the way if Pd does not sync with an audio device for whatever reason, you get weird test results anyhow. Even if you do not want to actually hear the sound, the audio device must work well, for these signal object tests. That was one of the first things we observed in practice.
Katja