Hello,
Here is a .zip with an improved method for unit-testing of signal objects. Instead of a template, there is now an abstraction 'unit-test-frame~.pd' handling the tests. The input of that abstraction consists of:
- name of the reference file (32 bit float .wav file, like Roman suggested) - signal (or system) under test - optional delay between test-trigger and test-snapshot (for filters etc.)
The reference file can simply be created from within the unit test patch (running in a 'good' Pd), and must be kept with the test and the abstraction in a directory. The abstraction compares signal under test with reference, computes standard deviation, and sends a tagged result to Pd window and stdout:
STDDEV-ZERO 0 reference: <reference> (when stddev = 0) STDDEV-SMALL <stddev> reference: <reference> (when 0 < stddev < 1e+05) STDDEV-LARGE <stddev> reference: <reference> (when stddev >= 1e+05)
The tags facilitate grepping different categories of 'errors' from a test log. Note that a small stddev may sometimes indicate improved accuracy in an object! Like when Pd-double is tested against current Pd.
By the way I did not find a way to set samplerate for a test. I definitely do not want the test patch to mess up other audio settings, like number of channels. Therefore an audio-dialog message to pd is not an option. However there's an inlet reserved on unit-test-frame~.pd for possible future solution of this issue. From a remark by Roman Haefeli in another thread, http://lists.puredata.info/pipermail/pd-dev/2011-10/017694.html, I learnt that -noaudio may be a useful option for automated signal object testing.
Please report if you have comments on the proposed test stuff. I should mention a hidden agenda behind the efforts: the intention is to produce dozends or even hundreds of unit-tests in short time, and use these for testing Pd-double. Then we could finally make some systematic progress in that field. Apart from that, the tests could be useful for rewrites/refactorings in general, and for tracking down optimizer-troubles.
Katja
.