I've been thinking now that we are starting to get some testing infrastructure in place, we need a place to organize them in the pure- data SVN. I propose we add a tests/ section to trunk in SVN next to externals/ scripts/ etc. This section would be for the testing scripts, and for any tests that don't specifically below to a library. Once we get a test format sorted out, I think that libraries should include their own tests.
I propose we use the suffix "-regressiontest.pd" for tests, then we can have a script that crawls the whole pure-data SVN and runs the tests in place with the libraries they are part of. (-test.pd is already used a lot of places, so a new suffix would be better to avoid confusion).
.hc
----------------------------------------------------------------------------
There is no way to peace, peace is the way. -A.J. Muste
On Mon, Oct 24, 2011 at 03:26:18PM -0400, Hans-Christoph Steiner wrote:
I've been thinking now that we are starting to get some testing infrastructure in place, we need a place to organize them in the pure- data SVN.
Have there been developments on this front? I haven't seen anything in the patch tracker messages. It was something I was going to work on, but I focused on UTF-8 first, then the Return/Control-Return/Escape interface issues. The interface issues turned out to be frustrating to work out, and I haven't gotten to the unit test libraries.
I propose we add a tests/ section to trunk in SVN next to externals/ scripts/ etc. This section would be for the testing scripts, and for any tests that don't specifically below to a library. Once we get a test format sorted out, I think that libraries should include their own tests.
Seems sane.
I propose we use the suffix "-regressiontest.pd" for tests, then we can have a script that crawls the whole pure-data SVN and runs the tests in place with the libraries they are part of. (-test.pd is already used a lot of places, so a new suffix would be better to avoid confusion).
That's clear, but long. Other possibilities:
-unittest.pd -t.pd .pdt .pdtest
Marvin Humphrey
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-10-24 22:10, Marvin Humphrey wrote:
I propose we use the suffix "-regressiontest.pd" for tests, then we can have a script that crawls the whole pure-data SVN and runs the tests in place with the libraries they are part of. (-test.pd is already used a lot of places, so a new suffix would be better to avoid confusion).
how about adding a "check" target (which i think is the standard target to run regression tests) to the template/Makefile, in order to run the actual tests?
e.g. $ make check -C zexy make check-TESTS make[2]: Entering directory `/Net/iem/Benutzer/zmoelnig/src/cvs/pure-data/externals/zexy/tests' running tests in . ./absgn~/abs: OK [...] ./or~/dualnoise: OK ./or~/singlenoise: OK ./repeat/noargs: OK ./s2l/numsymbols: OK ./sgn~/simple: OK ./testtests/fail_noresult: failed ./testtests/fail: failed ./testtests/fail_wait0: failed ./testtests/fail_wait10: failed ./testtests/pass: OK ./testtests/pass_wait0: OK ./testtests/pass_wait10: OK 101 tests passed 4 tests failed ====================================== 105 regression-tests total 105 regression-tests passed 0 regression-tests failed ====================================== PASS: runtests.sh ============= 1 test passed =============
On Monday, October 24, 2011 10:27 PM, "IOhannes m zmoelnig" zmoelnig@iem.at wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 2011-10-24 22:10, Marvin Humphrey wrote:
I propose we use the suffix "-regressiontest.pd" for tests, then we can have a script that crawls the whole pure-data SVN and runs the tests in place with the libraries they are part of. (-test.pd is already used a lot of places, so a new suffix would be better to avoid confusion).
how about adding a "check" target (which i think is the standard target to run regression tests) to the template/Makefile, in order to run the actual tests?
That makes sense, once we get this all nailed down, I'll do that. I think that this is not a replacement for the 'tests/' section tho. For example, it would be a place to develop tests on the core of Pd that would move far faster than the pure-data.git, and it would be a place for tests that involved multiple libraries.
.hc
On Monday, October 24, 2011 1:10 PM, "Marvin Humphrey" marvin@rectangular.com wrote:
On Mon, Oct 24, 2011 at 03:26:18PM -0400, Hans-Christoph Steiner wrote:
I've been thinking now that we are starting to get some testing infrastructure in place, we need a place to organize them in the pure- data SVN.
Have there been developments on this front? I haven't seen anything in the patch tracker messages. It was something I was going to work on, but I focused on UTF-8 first, then the Return/Control-Return/Escape interface issues. The interface issues turned out to be frustrating to work out, and I haven't gotten to the unit test libraries.
Katja and Fred Jan have put together a template that is looking pretty good, and I've nailed down the 'load_every_help.py' script so that it runs every night. Those are the developments I was talking about, but there is still much to be done, don't worry there, we would still love your help :)
Programming the GUI in Pd is indeed quite frustrating and in need of a major overhaul. Its about the worst place to get started for hacking on Pd.
I propose we add a tests/ section to trunk in SVN next to externals/ scripts/ etc. This section would be for the testing scripts, and for any tests that don't specifically below to a library. Once we get a test format sorted out, I think that libraries should include their own tests.
Seems sane.
I propose we use the suffix "-regressiontest.pd" for tests, then we can have a script that crawls the whole pure-data SVN and runs the tests in place with the libraries they are part of. (-test.pd is already used a lot of places, so a new suffix would be better to avoid confusion).
That's clear, but long. Other possibilities:
-unittest.pd -t.pd .pdt .pdtestMarvin Humphrey
I could go with -unittest.pd. I'm not really a fan of the rest. Adding new file extensions seems like extra management work. The tests should just be regular pd patches, so .pd should work. And we already have -help.pd and -meta.pd.
.hc
-
Hans-Christoph Steiner -
Hans-Christoph Steiner -
IOhannes m zmoelnig -
Marvin Humphrey