(moving this thread off of pd-list and onto pd-dev...)
On Wed, Sep 28, 2011 at 10:55:09AM -0400, Hans-Christoph Steiner wrote:
I don't see any unit tests. Are there any?
There isn't really any unified test method. I think the zexy library has some,
I looked at zexy's system to see what testing framework it used, in the hopes that Pd Vanilla could use the same one. But zexy is using its own custom rig and zexy is under the GPL, so I'm still looking for other solutions.
and someone else recently starting writing a test framework in Lua.
Hmm, it's always good to have something rather than nothing, but it seems desirable to have a test framework written in Tcl, C or Pd rather than to add Lua as a dependency.
Writing a simple testing framework for imperative code is easy. (I've written two and hacked on several.) Gui testing is much harder, so my initial thought is to just avoid it altogether and focus on imperative testing.
[ ... /me browses web, discovers tcltest ... ]
Hmm, wouldn't tcltest be the obvious framework to use?
http://www.tcl.tk/man/tcl/TclCmd/tcltest.htm
But regular automated unit tests is something that we sorely need.
In the abstract, I'm interested in this. Writing some test cases would be a great way for me to improve my skills with both Tcl and Pd.
Any suggestions about areas of Pd that are particularly suitable for unit testing?
What we do have is a farm of dev/build servers which you can get ssh access to.
Cool, that's a great resource!
Marvin Humphrey
On Sep 28, 2011, at 3:13 PM, Marvin Humphrey wrote:
(moving this thread off of pd-list and onto pd-dev...)
On Wed, Sep 28, 2011 at 10:55:09AM -0400, Hans-Christoph Steiner wrote:
I don't see any unit tests. Are there any?
There isn't really any unified test method. I think the zexy library has some,
I looked at zexy's system to see what testing framework it used, in the hopes that Pd Vanilla could use the same one. But zexy is using its own custom rig and zexy is under the GPL, so I'm still looking for other solutions.
and someone else recently starting writing a test framework in Lua.
Hmm, it's always good to have something rather than nothing, but it seems desirable to have a test framework written in Tcl, C or Pd rather than to add Lua as a dependency.
Writing a simple testing framework for imperative code is easy. (I've written two and hacked on several.) Gui testing is much harder, so my initial thought is to just avoid it altogether and focus on imperative testing.
[ ... /me browses web, discovers tcltest ... ]
Hmm, wouldn't tcltest be the obvious framework to use?
But regular automated unit tests is something that we sorely need.
In the abstract, I'm interested in this. Writing some test cases would be a great way for me to improve my skills with both Tcl and Pd.
Any suggestions about areas of Pd that are particularly suitable for unit testing?
I think that audio tests would be the best way to start. Basically generate some audio to an table/array, then compare it to the expected result. There are a number of objects that do type-punning to speed things up. Modern compilers with their vectorization and other optimziations don't like type punning, so its a common source of errors.
.hc
What we do have is a farm of dev/build servers which you can get ssh access to.
Cool, that's a great resource!
Marvin Humphrey
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
----------------------------------------------------------------------------
On Wed, Sep 28, 2011 at 04:38:30PM -0400, Hans-Christoph Steiner wrote:
But regular automated unit tests is something that we sorely need.
In the abstract, I'm interested in this. Writing some test cases would be a great way for me to improve my skills with both Tcl and Pd.
Any suggestions about areas of Pd that are particularly suitable for unit testing?
I think that audio tests would be the best way to start. Basically generate some audio to an table/array, then compare it to the expected result. There are a number of objects that do type-punning to speed things up. Modern compilers with their vectorization and other optimziations don't like type punning, so its a common source of errors.
Note that I created a bunch of unit tests with text file outputs for some of the non-audio parts of Pd for the WebPd project (for my own sanity):
http://mccormick.cx/projects/WebPd/tests
Note sure if they are in a state that is useful to anybody though.
Cheers,
Chris.
------------------- http://mccormick.cx
On Thu, Sep 29, 2011 at 08:29:05AM +0800, Chris McCormick wrote:
Note that I created a bunch of unit tests with text file outputs for some of the non-audio parts of Pd for the WebPd project (for my own sanity):
http://mccormick.cx/projects/WebPd/tests
Note sure if they are in a state that is useful to anybody though.
I see that they are AGPL licensed. I will respect your licensing choice and refrain from using any of your code in any BSD3-licensed patches I prepare.
Marvin Humphrey
----- Original Message -----
From: Marvin Humphrey marvin@rectangular.com To: Chris McCormick chris@mccormick.cx Cc: pd-dev@iem.at Sent: Wednesday, September 28, 2011 8:56 PM Subject: Re: [PD-dev] Unit tests
On Thu, Sep 29, 2011 at 08:29:05AM +0800, Chris McCormick wrote:
Note that I created a bunch of unit tests with text file outputs for some
of
the non-audio parts of Pd for the WebPd project (for my own sanity):
http://mccormick.cx/projects/WebPd/tests
Note sure if they are in a state that is useful to anybody though.
I see that they are AGPL licensed. I will respect your licensing choice and refrain from using any of your code in any BSD3-licensed patches I prepare.
Marvin Humphrey
Also, you might have a look at: http://artengine.ca/~catalogue-pd/43-Bouchard.pdf
I think there are some examples with DesireData: http://artengine.ca/desiredata/download/
have a look at extra/pureunity/README
-Jonathan
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
----- Original Message -----
From: Marvin Humphrey marvin@rectangular.com To: Chris McCormick chris@mccormick.cx Cc: pd-dev@iem.at Sent: Wednesday, September 28, 2011 8:56 PM Subject: Re: [PD-dev] Unit tests
On Thu, Sep 29, 2011 at 08:29:05AM +0800, Chris McCormick wrote:
Note that I created a bunch of unit tests with text file outputs for some
of
the non-audio parts of Pd for the WebPd project (for my own sanity):
http://mccormick.cx/projects/WebPd/tests
Note sure if they are in a state that is useful to anybody though.
I see that they are AGPL licensed. I will respect your licensing choice and refrain from using any of your code in any BSD3-licensed patches I prepare.
Marvin Humphrey
Also, you might have a look at: http://artengine.ca/~catalogue-pd/43-Bouchard.pdf
I think there are some examples with DesireData: http://artengine.ca/desiredata/download/
have a look at extra/pureunity/README
It appears that pureunity is similarly licensed under the GPL and would suffer the same objections from Marvin.
Note that Mathieu's unit testing implementation is almost certainly more rigorous, better thought out, and exact than my own. :)
Cheers,
Chris.
------------------- http://mccormick.cx
----- Original Message -----
From: Chris McCormick chris@mccormick.cx To: Jonathan Wilkes jancsika@yahoo.com Cc: Marvin Humphrey marvin@rectangular.com; "pd-dev@iem.at" pd-dev@iem.at Sent: Wednesday, September 28, 2011 9:58 PM Subject: Re: [PD-dev] Unit tests
----- Original Message -----
From: Marvin Humphrey marvin@rectangular.com To: Chris McCormick chris@mccormick.cx Cc: pd-dev@iem.at Sent: Wednesday, September 28, 2011 8:56 PM Subject: Re: [PD-dev] Unit tests
On Thu, Sep 29, 2011 at 08:29:05AM +0800, Chris McCormick wrote:
Note that I created a bunch of unit tests with text file outputs
for
some
of
the non-audio parts of Pd for the WebPd project (for my own
sanity):
http://mccormick.cx/projects/WebPd/tests
Note sure if they are in a state that is useful to anybody though.
I see that they are AGPL licensed. I will respect your licensing
choice
and refrain from using any of your code in any BSD3-licensed patches I prepare.
Marvin Humphrey
Also, you might have a look at: http://artengine.ca/~catalogue-pd/43-Bouchard.pdf
I think there are some examples with DesireData: http://artengine.ca/desiredata/download/
have a look at extra/pureunity/README
It appears that pureunity is similarly licensed under the GPL and would suffer the same objections from Marvin.
I forgot to address that part directly-- the author, Mathieu Bouchard matju@artengine.ca, is an active member of the Pd community and has stated several times on this list that he would be amenable to licensing code from DD under the 3-clause BSD license if someone were interested in having that functionality merged into the core of Pd.
So please, feel _free_ to have a look at the patches I listed above-- either you find them useful and discuss the licensing options with matju, or you will not find them useful in which case it doesn't matter.
If Stallman can convince large companies who sell licenses to also release the software under a free license, then surely an open community such as Pd can share code in a way less rigid than "IF $other_license THEN goto more_work". I'm not saying licenses aren't important, but I'm saying it's a non-problem when the original author is probably willing to give you what you need under the license you want in order to move Pd forward.
-Jonathan
Note that Mathieu's unit testing implementation is almost certainly more rigorous, better thought out, and exact than my own. :)
Cheers,
Chris.
I'm not saying licenses aren't important, but I'm saying it's a non-problem when the original author is probably willing to give you what you need under the license you want in order to move Pd forward.
Good point. I am now sending Marvin a copy of the WebPd unit test patches licensed under BSD-3.
Cheers,
Chris.
On Wed, Sep 28, 2011 at 11:08:49PM -0400, Chris McCormick wrote:
Good point. I am now sending Marvin a copy of the WebPd unit test patches licensed under BSD-3.
Cool!
I'm excited to look through these materials. I'm a Pd noob with a lot to learn and tests can serve as very enlightening documentation! :)
Sooner or later, I'll likely contribute unit tests, and I'm confident in my general testing expertise. I'm pleased to have started this thread... Still, it seems to me like the introduction of a testing framework ought to move ahead on its own, since there are already candidate frameworks available.
There are several long-time Pd community members who believe that the core would benefit from the addition of unit tests. I'd like to add my voice to their chorus.
I would also like to opine that the addition of a unit testing framework seems like it would be a conservative, non-disruptive change. A public API for the the testing framework need not be exposed -- not right away, and theoretically never. Existing core code should require few if any changes to accommodate a test harness. Naturally, though, we would want the framework we use to be the best it can be.
What opinions are out there with regards to the two existing candidate frameworks? What do they have in common? How do they contrast? Do they suggest improvements for each other?
Marvin Humphrey
On Sep 29, 2011, at 12:10 AM, Marvin Humphrey wrote:
On Wed, Sep 28, 2011 at 11:08:49PM -0400, Chris McCormick wrote:
Good point. I am now sending Marvin a copy of the WebPd unit test patches licensed under BSD-3.
Cool!
I'm excited to look through these materials. I'm a Pd noob with a lot to learn and tests can serve as very enlightening documentation! :)
Sooner or later, I'll likely contribute unit tests, and I'm confident in my general testing expertise. I'm pleased to have started this thread... Still, it seems to me like the introduction of a testing framework ought to move ahead on its own, since there are already candidate frameworks available.
There are several long-time Pd community members who believe that the core would benefit from the addition of unit tests. I'd like to add my voice to their chorus.
I would also like to opine that the addition of a unit testing framework seems like it would be a conservative, non-disruptive change. A public API for the the testing framework need not be exposed -- not right away, and theoretically never. Existing core code should require few if any changes to accommodate a test harness. Naturally, though, we would want the framework we use to be the best it can be.
What opinions are out there with regards to the two existing candidate frameworks? What do they have in common? How do they contrast? Do they suggest improvements for each other?
We've known for many years that tests would be a really useful thing. Its just a matter of someone doing the work. A few of us have done some work here and there, as you have seen. But we haven't gotten a coherent package. So I'll be happy to see you work on this :)
.hc
----------------------------------------------------------------------------
I have the audacity to believe that peoples everywhere can have three meals a day for their bodies, education and culture for their minds, and dignity, equality and freedom for their spirits. - Martin Luther King, Jr.
On Wed, Sep 28, 2011 at 11:08:49PM -0400, Chris McCormick wrote:
Good point. I am now sending Marvin a copy of the WebPd unit test patches licensed under BSD-3.
Cool!
I'm excited to look through these materials. I'm a Pd noob with a lot to learn and tests can serve as very enlightening documentation! :)
Argh, please hold on a second. I have just realised that I do not have the authority to do this. I should have asked two other people who contributed unit tests. Sorry about that, I will get back to you.
Cheers,
Chris.
------------------- http://mccormick.cx
On Thu, Sep 29, 2011 at 01:04:12AM -0400, Chris McCormick wrote:
Argh, please hold on a second. I have just realised that I do not have the authority to do this. I should have asked two other people who contributed unit tests. Sorry about that, I will get back to you.
You caught it before the release, which is the important thing! And before a commit, and before I even had the chance to look through it. So no harm done and thanks for staying on top of things!
It also sounds as if the other people may have contributed tests, but not touched the framework -- in which case, perhaps their IP could be omitted from the contribution if it turns out that either or both are not amenable to granting an additional license.
Marvin Humphrey
On Sep 29, 2011, at 1:27 AM, Marvin Humphrey wrote:
On Thu, Sep 29, 2011 at 01:04:12AM -0400, Chris McCormick wrote:
Argh, please hold on a second. I have just realised that I do not have the authority to do this. I should have asked two other people who contributed unit tests. Sorry about that, I will get back to you.
You caught it before the release, which is the important thing! And before a commit, and before I even had the chance to look through it. So no harm done and thanks for staying on top of things!
It also sounds as if the other people may have contributed tests, but not touched the framework -- in which case, perhaps their IP could be omitted from the contribution if it turns out that either or both are not amenable to granting an additional license.
Marvin Humphrey
Can I ask, why are pushing the BSD license? I am curious because it seems you are more interested in convincing people to use the BSD license than anything else in Pd so far.
.hc
----------------------------------------------------------------------------
¡El pueblo unido jamás será vencido!
----- Original Message -----
From: Hans-Christoph Steiner hans@at.or.at To: Marvin Humphrey marvin@rectangular.com Cc: Chris McCormick chris@mccormick.cx; "pd-dev@iem.at" pd-dev@iem.at Sent: Thursday, September 29, 2011 3:56 PM Subject: Re: [PD-dev] Unit tests
On Sep 29, 2011, at 1:27 AM, Marvin Humphrey wrote:
On Thu, Sep 29, 2011 at 01:04:12AM -0400, Chris McCormick wrote:
Argh, please hold on a second. I have just realised that I do not have
the
authority to do this. I should have asked two other people who
contributed
unit tests. Sorry about that, I will get back to you.
You caught it before the release, which is the important thing! And before
a
commit, and before I even had the chance to look through it. So no harm
done
and thanks for staying on top of things!
It also sounds as if the other people may have contributed tests, but not touched the framework -- in which case, perhaps their IP could be omitted
from
the contribution if it turns out that either or both are not amenable to granting an additional license.
Marvin Humphrey
Can I ask, why are pushing the BSD license? I am curious because it seems you are more interested in convincing people to use the BSD license than anything else in Pd so far.
I don't think it matters at all and would like to refrain from getting into a list argument about the pluses and minuses of all the licenses used by Pd and its externals. Pd Vanilla is 3-clause BSD, and according to IOhannes so is the stuff on the patch tracker. That gives plenty of room for someone like Marvin to develop solely using the 3-clause BSD license.
I would only say that the valid constraint of using only one free license shouldn't add a constant burden to other developers who are fine with the many free licenses that exist in Pd and its external libraries.
-Jonathan
.hc
¡El pueblo unido jamás será vencido!
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 09/29/2011 10:43 PM, Jonathan Wilkes wrote:
Pd Vanilla is 3-clause BSD, and according to IOhannes so is the stuff on the patch tracker.
just for clarification: i did not intend to say that. instead i wanted to say that "stuff" in the patch tracker is (assumed to have) the same license as the code it is applied to, unless explicitely stated.
mfgasdr IOhannes
PS: some more examples (though it's probably clear anyhow)
if you create a patch for Pd-vanilla and you don't mention a special license anywhere, you are implicitly licensing your stuff under BSD-3 (Pd-vanilla's license) if you create a patch for zexy, you are implicitely licensing your stuff under GPL-2+.
if you create a patch for Pd-extended, things are a bit more complicated, and i would assume that the implicit license applies to the part you are actually patching. there is hardly any code that is Pd-extended specific, as PdX is mainly a collection of loads of packages. adding to the confusion is, that there are patches (form the patch-tracker) that are in Pd-extended but not "upstream" (Pd-vanilla or some externals) e.g. my "initbang" patch is included (afaik) in PdX but not vanilla, but it is really a patch for vanilla and thus implicitely licensed under BSD-3 rather than GPL-3+.
On Wed, Sep 28, 2011 at 07:30:23PM -0700, Jonathan Wilkes wrote:
I forgot to address that part directly-- the author, Mathieu Bouchard matju@artengine.ca, is an active member of the Pd community and has stated several times on this list that he would be amenable to licensing code from DD under the 3-clause BSD license if someone were interested in having that functionality merged into the core of Pd.
Phew! :)
I'm looking forward to spelunking this code!
So please, feel _free_ to have a look at the patches I listed above-- either you find them useful and discuss the licensing options with matju, or you will not find them useful in which case it doesn't matter.
For the sake of due diligence, and because I've been through a relicensing myself that involved ~20 parties... Relicensing is a major PITA when there are multiple entities holding a copyright claim on the code. Is there truly only one author for the DesireData test harness?
Marvin Humphrey
----- Original Message -----
From: Marvin Humphrey marvin@rectangular.com To: Jonathan Wilkes jancsika@yahoo.com Cc: Chris McCormick chris@mccormick.cx; "pd-dev@iem.at" pd-dev@iem.at Sent: Thursday, September 29, 2011 12:24 AM Subject: Re: [PD-dev] Unit tests
On Wed, Sep 28, 2011 at 07:30:23PM -0700, Jonathan Wilkes wrote:
I forgot to address that part directly-- the author, Mathieu Bouchard matju@artengine.ca, is an active member of the Pd community and has
stated
several times on this list that he would be amenable to licensing code from DD under the 3-clause BSD license if someone were interested in having that functionality merged into the core of Pd.
Phew! :)
I'm looking forward to spelunking this code!
So please, feel _free_ to have a look at the patches I listed above--
either you
find them useful and discuss the licensing options with matju, or you will
not
find them useful in which case it doesn't matter.
For the sake of due diligence, and because I've been through a relicensing myself that involved ~20 parties... Relicensing is a major PITA when there are multiple entities holding a copyright claim on the code. Is there truly only one author for the DesireData test harness?
Yes.
Marvin Humphrey
On Wed, Sep 28, 2011 at 04:38:30PM -0400, Hans-Christoph Steiner wrote:
I think that audio tests would be the best way to start. Basically generate some audio to an table/array, then compare it to the expected result. There are a number of objects that do type-punning to speed things up. Modern compilers with their vectorization and other optimziations don't like type punning, so its a common source of errors.
Interesting. Before I entered the field of software development, I spent some time working as an audio mastering engineer, and we used to do "null tests" to validate the integrity of digital audio copies which involved flipping polarity on the copy and confirming that adding that to the original resulted in a flatline. Writing these tests will be nostalgic. :)
After some research today, I think it's going to take me a little while to work out the testing framework, though. Perhaps I'll stop by the IRC channel for some brainstorming.
Marvin Humphrey
-
Chris McCormick -
Hans-Christoph Steiner -
IOhannes m zmölnig -
Jonathan Wilkes -
Marvin Humphrey