Hey Andras,
I think the next step would be getting buildbot building pure-data.git. That's where the active development currently is. Also, I'll set up the Mac build servers with buildbot and give you ssh access, if you can take it from there.
This is awesome, I'm happy to see this starting to work!
.hc
Hey Andras,
Ok 128.238.56.59 (macosx104-i386) is setup with a buildbot user with your ssh key, and buildbot is setup as launchd service: http://buildbot.net/trac/wiki/UsingLaunchd
The file is here, let me know if you need me to change it (probably): /Library/LaunchDaemons/net.sourceforge.buildbot.plist
141.54.159.89 (macosx106-x86_64) won't work yet, it seems that twisted2 doesn't build on 64-bit yet.
.hc
On Sep 13, 2010, at 6:01 PM, Hans-Christoph Steiner wrote:
Hey Andras,
I think the next step would be getting buildbot building pure- data.git. That's where the active development currently is. Also, I'll set up the Mac build servers with buildbot and give you ssh access, if you can take it from there.
This is awesome, I'm happy to see this starting to work!
.hc
----------------------------------------------------------------------------
All mankind is of one author, and is one volume; when one man dies, one chapter is not torn out of the book, but translated into a better language; and every chapter must be so translated.... -John Donne
On Tue, Sep 14, 2010 at 1:16 AM, Hans-Christoph Steiner hans@at.or.atwrote:
Hey Andras,
Ok 128.238.56.59 (macosx104-i386) is setup with a buildbot user with your ssh key, and buildbot is setup as launchd service: http://buildbot.net/trac/wiki/UsingLaunchd
Aww, heavy reading! How can i run the 'buildbot create-slave' command just once?
The file is here, let me know if you need me to change it (probably): /Library/LaunchDaemons/net.sourceforge.buildbot.plist
It seems only this... <key>WorkingDirectory</key> <string>/Users/buildbot/slave/test</string> ...needs to have the right path which shall be /Users/buildbot/macosx104-i386 once the slave is created.
If it cannot be run manually, we can try to fake it by copying (and modding) the files from another slave, shall we?
Andras
141.54.159.89 (macosx106-x86_64) won't work yet, it seems that twisted2 doesn't build on 64-bit yet.
.hc
On Sep 13, 2010, at 6:01 PM, Hans-Christoph Steiner wrote:
Hey Andras,
I think the next step would be getting buildbot building pure-data.git. That's where the active development currently is. Also, I'll set up the Mac build servers with buildbot and give you ssh access, if you can take it from there.
This is awesome, I'm happy to see this starting to work!
.hc
All mankind is of one author, and is one volume; when one man dies, one chapter is not torn out of the book, but translated into a better language; and every chapter must be so translated.... -John Donne
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
On Sep 13, 2010, at 8:48 PM, András Murányi wrote:
On Tue, Sep 14, 2010 at 1:16 AM, Hans-Christoph Steiner hans@at.or.at wrote:
Hey Andras,
Ok 128.238.56.59 (macosx104-i386) is setup with a buildbot user with your ssh key, and buildbot is setup as launchd service: http://buildbot.net/trac/wiki/UsingLaunchd
Aww, heavy reading! How can i run the 'buildbot create-slave' command just once?
You can run it in the buildbot account using:
sudo /sw/bin/buildbot create-slave
The file is here, let me know if you need me to change it (probably): /Library/LaunchDaemons/net.sourceforge.buildbot.plist
It seems only this... <key>WorkingDirectory</key> <string>/Users/buildbot/slave/test</string> ...needs to have the right path which shall be /Users/buildbot/ macosx104-i386 once the slave is created.
If it cannot be run manually, we can try to fake it by copying (and modding) the files from another slave, shall we?
Ok, I edited it, and gave 'buildbot' rw perms, so you can edit if need be. Let me know when you are done so I can convert it back to being owned by root.
.hc
Andras
141.54.159.89 (macosx106-x86_64) won't work yet, it seems that twisted2 doesn't build on 64-bit yet.
.hc
On Sep 13, 2010, at 6:01 PM, Hans-Christoph Steiner wrote:
Hey Andras,
I think the next step would be getting buildbot building pure- data.git. That's where the active development currently is. Also, I'll set up the Mac build servers with buildbot and give you ssh access, if you can take it from there.
This is awesome, I'm happy to see this starting to work!
.hc
All mankind is of one author, and is one volume; when one man dies, one chapter is not torn out of the book, but translated into a better language; and every chapter must be so translated.... -John Donne
Pd-dev mailing list Pd-dev@iem.at http://lists.puredata.info/listinfo/pd-dev
-- Muranyi Andras
----------------------------------------------------------------------------
Information wants to be free. -Stewart Brand
Ok 128.238.56.59 (macosx104-i386) is setup with a buildbot user with your ssh key, and buildbot is setup as launchd service: http://buildbot.net/trac/wiki/UsingLaunchd
Aww, heavy reading! How can i run the 'buildbot create-slave' command just once?
You can run it in the buildbot account using:
sudo /sw/bin/buildbot create-slave
done, up and running
The file is here, let me know if you need me to change it (probably):
/Library/LaunchDaemons/net.sourceforge.buildbot.plist
It seems only this... <key>WorkingDirectory</key> <string>/Users/buildbot/slave/test</string> ...needs to have the right path which shall be /Users/buildbot/macosx104-i386 once the slave is created.
If it cannot be run manually, we can try to fake it by copying (and modding) the files from another slave, shall we?
Ok, I edited it, and gave 'buildbot' rw perms, so you can edit if need be. Let me know when you are done so I can convert it back to being owned by root.
.hc
This is done too. At the moment buildbot fails with "exceptions.RuntimeError: Couldn't find executable for 'svn'"
Andras
On Sep 14, 2010, at 4:33 PM, András Murányi wrote:
At the moment buildbot fails with "exceptions.RuntimeError: Couldn't find executable for 'svn'"
- the same for git
I added ~buildbot/.bashrc to hopefully add Fink stuff to buildbot's environment
.hc
----------------------------------------------------------------------------
"Free software means you control what your computer does. Non-free software means someone else controls that, and to some extent controls you." - Richard M. Stallman
At the moment buildbot fails with "exceptions.RuntimeError: Couldn't find
executable for 'svn'"
- the same for git
I added ~buildbot/.bashrc to hopefully add Fink stuff to buildbot's environment
.hc
Hmm, didn't work out, and net.sourceforge.buildbot.plist has sw/bin in the path too, and svn is there, so i really don't know...
Andras
On Sep 14, 2010, at 5:08 PM, András Murányi wrote:
At the moment buildbot fails with "exceptions.RuntimeError: Couldn't find executable for 'svn'"
- the same for git
I added ~buildbot/.bashrc to hopefully add Fink stuff to buildbot's environment
.hc
Hmm, didn't work out, and net.sourceforge.buildbot.plist has sw/bin in the path too, and svn is there, so i really don't know...
Andras
Ok, I was getting some builds from macosx104-i386, but then it disappeared, donno what happened there. macosx104-powerpc seems to be running still tho.
I worked on the pd-master/master.cfg a bit, including changing some of the names to be more consistent. I also got pure-data building from Miller's git.
http://128.238.56.50:8010/builders/pure-data%20Linux%20debian-stable-i386/bu...
.hc
----------------------------------------------------------------------------
"[W]e have invented the technology to eliminate scarcity, but we are deliberately throwing it away to benefit those who profit from scarcity." -John Gilmore
2010/9/15 Hans-Christoph Steiner hans@at.or.at
On Sep 14, 2010, at 5:08 PM, András Murányi wrote:
At the moment buildbot fails with "exceptions.RuntimeError: Couldn't find executable for 'svn'"
- the same for git
I added ~buildbot/.bashrc to hopefully add Fink stuff to buildbot's environment
.hc
Hmm, didn't work out, and net.sourceforge.buildbot.plist has sw/bin in the path too, and svn is there, so i really don't know...
Andras
Ok, I was getting some builds from macosx104-i386, but then it disappeared, donno what happened there. macosx104-powerpc seems to be running still tho.
Yea the process died somehow. I have restarted it with the command: /sw/bin/buildbot restart /Users/buildbot/macosx104-i386
The output had some complaints: /sw/lib/python2.6/site-packages/twisted/persisted/sob.py:12: DeprecationWarning: the md5 module is deprecated; use hashlib instead import os, md5, sys /sw/lib/python2.6/site-packages/twisted/python/filepath.py:12: DeprecationWarning: the sha module is deprecated; use the hashlib module instead import sha /sw/lib/python2.6/site-packages/twisted/internet/_sslverify.py:5: DeprecationWarning: the md5 module is deprecated; use hashlib instead import itertools, md5 Following twistd.log until startup finished.. /sw/lib/python2.6/site-packages/buildbot/scripts/logwatcher.py:52: PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler is not installed. This probably means you have not yet called reactor.run, or called reactor.run(installSignalHandler=0). You will probably never see this process finish, and it may become a zombie process. env=os.environ, Removing stale pidfile /Users/buildbot/macosx104-i386/twistd.pid
I worked on the pd-master/master.cfg a bit, including changing some of the names to be more consistent. I also got pure-data building from Miller's git.
http://128.238.56.50:8010/builders/pure-data%20Linux%20debian-stable-i386/bu...
.hc
Good! I saw you stared experimenting with a builder for the externals too - do i understand right that at the end we will have every external built separately? I was thinking about breaking them out to a separate master, but then we'd need to duplicate every slave setup, so finally i think they could stay in the main master, and we could have each of their have their own "category" name, which allows for some selection at the web page. Also note that for the builders, you can define an array with "slavenames:" instead of a single string "slavename", so you can test the same builder on multiple slaves at the same time. At the end they have to broken down to one slave per builder, otherwise the diag output is not easy to understand. Changing descriptionDone values to past tense like "compiled" may not make sense when the step fails and the output goes like "compiled failed". Also there are things like "autogen" which don't have a proper past tense... :) Another thing i noticed an "svn update" by itself, i think we shall have the sources explicitly in master.cfg otherwise it will fail where the slave got reset. Also you told before we wanted "clobber" (tabula rasa) checkout not an update... I saw the "make install", "make uninstall" steps in the output - having these would make much sense, fyi tests can be called with Test() which has some advantages over ShellCommand() like it doesn't make the whole build halt on failure. BTW switching the sources to git is easy, what we have to work out is Git polling. It's built into 0.8.1 but needs to triggered from git for 0.7.12. And then we have this thing with the poller to explain it which builder to start upon updates... a.k.a. the cake :)
Andras
On Wed, 2010-09-15 at 22:11 +0200, András Murányi wrote:
2010/9/15 Hans-Christoph Steiner hans@at.or.at
On Sep 14, 2010, at 5:08 PM, András Murányi wrote: At the moment buildbot fails with "exceptions.RuntimeError: Couldn't find executable for 'svn'" + the same for git I added ~buildbot/.bashrc to hopefully add Fink stuff to buildbot's environment .hc Hmm, didn't work out, and net.sourceforge.buildbot.plist has sw/bin in the path too, and svn is there, so i really don't know... Andras Ok, I was getting some builds from macosx104-i386, but then it disappeared, donno what happened there. macosx104-powerpc seems to be running still tho.Yea the process died somehow. I have restarted it with the command: /sw/bin/buildbot restart /Users/buildbot/macosx104-i386
The output had some complaints: /sw/lib/python2.6/site-packages/twisted/persisted/sob.py:12: DeprecationWarning: the md5 module is deprecated; use hashlib instead import os, md5, sys /sw/lib/python2.6/site-packages/twisted/python/filepath.py:12: DeprecationWarning: the sha module is deprecated; use the hashlib module instead import sha /sw/lib/python2.6/site-packages/twisted/internet/_sslverify.py:5: DeprecationWarning: the md5 module is deprecated; use hashlib instead import itertools, md5 Following twistd.log until startup finished.. /sw/lib/python2.6/site-packages/buildbot/scripts/logwatcher.py:52: PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler is not installed. This probably means you have not yet called reactor.run, or called reactor.run(installSignalHandler=0). You will probably never see this process finish, and it may become a zombie process. env=os.environ, Removing stale pidfile /Users/buildbot/macosx104-i386/twistd.pid
I worked on the pd-master/master.cfg a bit, including changing some of the names to be more consistent. I also got pure-data building from Miller's git. http://128.238.56.50:8010/builders/pure-data%20Linux% 20debian-stable-i386/builds/6 .hcGood! I saw you stared experimenting with a builder for the externals too - do i understand right that at the end we will have every external built separately? I was thinking about breaking them out to a separate master, but then we'd need to duplicate every slave setup, so finally i think they could stay in the main master, and we could have each of their have their own "category" name, which allows for some selection at the web page. Also note that for the builders, you can define an array with "slavenames:" instead of a single string "slavename", so you can test the same builder on multiple slaves at the same time. At the end they have to broken down to one slave per builder, otherwise the diag output is not easy to understand.
Changing descriptionDone values to past tense like "compiled" may not make sense when the step fails and the output goes like "compiled failed". Also there are things like "autogen" which don't have a proper past tense... :) Another thing i noticed an "svn update" by itself, i think we shall have the sources explicitly in master.cfg otherwise it will fail where the slave got reset. Also you told before we wanted "clobber" (tabula rasa) checkout not an update... I saw the "make install", "make uninstall" steps in the output - having these would make much sense, fyi tests can be called with Test() which has some advantages over ShellCommand() like it doesn't make the whole build halt on failure. BTW switching the sources to git is easy, what we have to work out is Git polling. It's built into 0.8.1 but needs to triggered from git for 0.7.12. And then we have this thing with the poller to explain it which builder to start upon updates... a.k.a. the cake :)
Andras
One approach would be to use multiple PBChangeSource things and have commit hooks report to buildbot that they should build:
http://stackoverflow.com/questions/2795386/support-for-multiple-repositories...
Another promising thing is loki, a web interface for easily setting up master/slaves:
https://fedorahosted.org/loki/
.hc
At the moment buildbot fails with "exceptions.RuntimeError: Couldn't find executable for 'svn'" + the same for git I added ~buildbot/.bashrc to hopefully add Fink stuff to buildbot's environment .hc Hmm, didn't work out, and net.sourceforge.buildbot.plist has sw/bin in the path too, and svn is there, so i really don't know... Andras Ok, I was getting some builds from macosx104-i386, but then it disappeared, donno what happened there. macosx104-powerpc seems to be running still tho.Yea the process died somehow. I have restarted it with the command: /sw/bin/buildbot restart /Users/buildbot/macosx104-i386
The output had some complaints: /sw/lib/python2.6/site-packages/twisted/persisted/sob.py:12: DeprecationWarning: the md5 module is deprecated; use hashlib instead import os, md5, sys /sw/lib/python2.6/site-packages/twisted/python/filepath.py:12: DeprecationWarning: the sha module is deprecated; use the hashlib module instead import sha /sw/lib/python2.6/site-packages/twisted/internet/_sslverify.py:5: DeprecationWarning: the md5 module is deprecated; use hashlib instead import itertools, md5 Following twistd.log until startup finished.. /sw/lib/python2.6/site-packages/buildbot/scripts/logwatcher.py:52: PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler is not installed. This probably means you have not yet called reactor.run, or called reactor.run(installSignalHandler=0). You will probably never see this process finish, and it may become a zombie process. env=os.environ, Removing stale pidfile /Users/buildbot/macosx104-i386/twistd.pid
I worked on the pd-master/master.cfg a bit, including changing some of the names to be more consistent. I also got pure-data building from Miller's git. http://128.238.56.50:8010/builders/pure-data%20Linux% 20debian-stable-i386/builds/6 .hcGood! I saw you stared experimenting with a builder for the externals too - do i understand right that at the end we will have every external built separately? I was thinking about breaking them out to a separate master, but then we'd need to duplicate every slave setup, so finally i think they could stay in the main master, and we could have each of their have their own "category" name, which allows for some selection at the web page. Also note that for the builders, you can define an array with "slavenames:" instead of a single string "slavename", so you can test the same builder on multiple slaves at the same time. At the end they have to broken down to one slave per builder, otherwise the diag output is not easy to understand.
Changing descriptionDone values to past tense like "compiled" may not make sense when the step fails and the output goes like "compiled failed". Also there are things like "autogen" which don't have a proper past tense... :) Another thing i noticed an "svn update" by itself, i think we shall have the sources explicitly in master.cfg otherwise it will fail where the slave got reset. Also you told before we wanted "clobber" (tabula rasa) checkout not an update... I saw the "make install", "make uninstall" steps in the output - having these would make much sense, fyi tests can be called with Test() which has some advantages over ShellCommand() like it doesn't make the whole build halt on failure. BTW switching the sources to git is easy, what we have to work out is Git polling. It's built into 0.8.1 but needs to triggered from git for 0.7.12. And then we have this thing with the poller to explain it which builder to start upon updates... a.k.a. the cake :)
Andras
One approach would be to use multiple PBChangeSource things and have commit hooks report to buildbot that they should build:
Actually there is only one PBChangeSource needed/possible because it's a listener, and multiple commit hooks can communicate with it from the repos.
Another promising thing is loki, a web interface for easily setting up master/slaves:
Hm. Seems a bit young to me...
I see that at the moment every factory is "under construcion" so i won't touch the config file for a while so that you can edit around. (Which makes me think about putting it in svn later...) http://128.238.56.50:8010/one_box_per_builder
Andras
On Thu, 2010-09-16 at 02:31 +0200, András Murányi wrote:
> > At the moment buildbot fails with > "exceptions.RuntimeError: Couldn't find executable for > 'svn'" > > + the same for git > > I added ~buildbot/.bashrc to hopefully add Fink stuff > to buildbot's environment > > .hc > > > Hmm, didn't work out, and > net.sourceforge.buildbot.plist has sw/bin in the path > too, and svn is there, so i really don't know... > > Andras > > > Ok, I was getting some builds from macosx104-i386, but then it > disappeared, donno what happened there. macosx104-powerpc > seems to be running still tho. > > > Yea the process died somehow. I have restarted it with the command: > /sw/bin/buildbot restart /Users/buildbot/macosx104-i386 > > The output had some complaints: > /sw/lib/python2.6/site-packages/twisted/persisted/sob.py:12: > DeprecationWarning: the md5 module is deprecated; use hashlib instead > import os, md5, sys > /sw/lib/python2.6/site-packages/twisted/python/filepath.py:12: > DeprecationWarning: the sha module is deprecated; use the hashlib > module instead > import sha > /sw/lib/python2.6/site-packages/twisted/internet/_sslverify.py:5: > DeprecationWarning: the md5 module is deprecated; use hashlib instead > import itertools, md5 > Following twistd.log until startup finished.. > /sw/lib/python2.6/site-packages/buildbot/scripts/logwatcher.py:52: > PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler > is not installed. This probably means you have not yet called > reactor.run, or called reactor.run(installSignalHandler=0). You will > probably never see this process finish, and it may become a zombie > process. > env=os.environ, > Removing stale pidfile /Users/buildbot/macosx104-i386/twistd.pid > > I worked on the pd-master/master.cfg a bit, including changing > some of the names to be more consistent. I also got pure-data > building from Miller's git. > > http://128.238.56.50:8010/builders/pure-data%20Linux % > 20debian-stable-i386/builds/6 > > .hc > > > Good! I saw you stared experimenting with a builder for the externals > too - do i understand right that at the end we will have every > external built separately? I was thinking about breaking them out to a > separate master, but then we'd need to duplicate every slave setup, so > finally i think they could stay in the main master, and we could have > each of their have their own "category" name, which allows for some > selection at the web page. > Also note that for the builders, you can define an array with > "slavenames:" instead of a single string "slavename", so you can test > the same builder on multiple slaves at the same time. At the end they > have to broken down to one slave per builder, otherwise the diag > output is not easy to understand. > > Changing descriptionDone values to past tense like "compiled" may not > make sense when the step fails and the output goes like "compiled > failed". Also there are things like "autogen" which don't have a > proper past tense... :) > Another thing i noticed an "svn update" by itself, i think we shall > have the sources explicitly in master.cfg otherwise it will fail where > the slave got reset. Also you told before we wanted "clobber" (tabula > rasa) checkout not an update... > I saw the "make install", "make uninstall" steps in the output - > having these would make much sense, fyi tests can be called with > Test() which has some advantages over ShellCommand() like it doesn't > make the whole build halt on failure. > BTW switching the sources to git is easy, what we have to work out is > Git polling. It's built into 0.8.1 but needs to triggered from git for > 0.7.12. And then we have this thing with the poller to explain it > which builder to start upon updates... a.k.a. the cake :) > > Andras One approach would be to use multiple PBChangeSource things and have commit hooks report to buildbot that they should build:Actually there is only one PBChangeSource needed/possible because it's a listener, and multiple commit hooks can communicate with it from the repos.
Another promising thing is loki, a web interface for easily setting up master/slaves: https://fedorahosted.org/loki/Hm. Seems a bit young to me...
I see that at the moment every factory is "under construcion" so i won't touch the config file for a while so that you can edit around. (Which makes me think about putting it in svn later...) http://128.238.56.50:8010/one_box_per_builder
Andras
I'm actually not going to have any time to look at this for a couple weeks, so please edit away. About the 'externals' collection, one idea I had was to set up a different step for each library, then have a common collection of steps for easy reuse. For the Makefile-based libs the steps could be: make, make install, make dist. For autotools, we could have ./configure, make, make install. If it uses automake, then these should also be available: make uninstall, make dist.
Perhaps that would be manageable, but it seems there might be a better way.
.hc
> At the moment buildbot fails with > "exceptions.RuntimeError: Couldn't find executable for > 'svn'" > > + the same for git > > I added ~buildbot/.bashrc to hopefully add Fink stuff > to buildbot's environment > > .hc > > > Hmm, didn't work out, and > net.sourceforge.buildbot.plist has sw/bin in the path > too, and svn is there, so i really don't know... > > Andras > > > Ok, I was getting some builds from macosx104-i386, but then it > disappeared, donno what happened there. macosx104-powerpc > seems to be running still tho. > > > Yea the process died somehow. I have restarted it with the command: > /sw/bin/buildbot restart /Users/buildbot/macosx104-i386 > > The output had some complaints: > /sw/lib/python2.6/site-packages/twisted/persisted/sob.py:12: > DeprecationWarning: the md5 module is deprecated; use hashlib instead > import os, md5, sys > /sw/lib/python2.6/site-packages/twisted/python/filepath.py:12: > DeprecationWarning: the sha module is deprecated; use the hashlib > module instead > import sha >/sw/lib/python2.6/site-packages/twisted/internet/_sslverify.py:5:
> DeprecationWarning: the md5 module is deprecated; use hashlib instead > import itertools, md5 > Following twistd.log until startup finished.. >/sw/lib/python2.6/site-packages/buildbot/scripts/logwatcher.py:52:
> PotentialZombieWarning: spawnProcess called, but the SIGCHLD handler > is not installed. This probably means you have not yet called > reactor.run, or called reactor.run(installSignalHandler=0). You will > probably never see this process finish, and it may become a zombie > process. > env=os.environ, > Removing stale pidfile /Users/buildbot/macosx104-i386/twistd.pid > > I worked on the pd-master/master.cfg a bit, including changing > some of the names to be more consistent. I also got pure-data > building from Miller's git. > > http://128.238.56.50:8010/builders/pure-data%20Linux % > 20debian-stable-i386/builds/6 > > .hc > > > Good! I saw you stared experimenting with a builder for the externals > too - do i understand right that at the end we will have every > external built separately? I was thinking about breaking them out to a > separate master, but then we'd need to duplicate every slave setup, so > finally i think they could stay in the main master, and we could have > each of their have their own "category" name, which allows for some > selection at the web page. > Also note that for the builders, you can define an array with > "slavenames:" instead of a single string "slavename", so you can test > the same builder on multiple slaves at the same time. At the end they > have to broken down to one slave per builder, otherwise the diag > output is not easy to understand. > > Changing descriptionDone values to past tense like "compiled" may not > make sense when the step fails and the output goes like "compiled > failed". Also there are things like "autogen" which don't have a > proper past tense... :) > Another thing i noticed an "svn update" by itself, i think we shall > have the sources explicitly in master.cfg otherwise it will fail where > the slave got reset. Also you told before we wanted "clobber" (tabula > rasa) checkout not an update... > I saw the "make install", "make uninstall" steps in the output - > having these would make much sense, fyi tests can be called with > Test() which has some advantages over ShellCommand() like it doesn't > make the whole build halt on failure. > BTW switching the sources to git is easy, what we have to work out is > Git polling. It's built into 0.8.1 but needs to triggered from git for > 0.7.12. And then we have this thing with the poller to explain it > which builder to start upon updates... a.k.a. the cake :) > > Andras One approach would be to use multiple PBChangeSource things and have commit hooks report to buildbot that they should build:Actually there is only one PBChangeSource needed/possible because it's a listener, and multiple commit hooks can communicate with it from the repos.
Another promising thing is loki, a web interface for easily setting up master/slaves: https://fedorahosted.org/loki/Hm. Seems a bit young to me...
I see that at the moment every factory is "under construcion" so i won't touch the config file for a while so that you can edit around. (Which makes me think about putting it in svn later...) http://128.238.56.50:8010/one_box_per_builder
Andras
I'm actually not going to have any time to look at this for a couple weeks, so please edit away.
Ok, i will. I'm not very good at build building generally, and which steps certain pd sources need, but i'll experiment. I'll bug you if something fails and i really don't know why.
About the 'externals' collection, one idea I had was to set up a different step for each library, then have a common collection of steps for easy reuse. For the Makefile-based libs the steps could be: make, make install, make dist. For autotools, we could have ./configure, make, make install. If it uses automake, then these should also be available: make uninstall, make dist.
Perhaps that would be manageable, but it seems there might be a better way.
Sounds good to me. If i set up that failed libs don't halt the whole build, it might be a good solution, or i'll think if there's a better one.
Andras
On Sep 16, 2010, at 12:21 PM, András Murányi wrote:
> At the moment buildbot fails with > "exceptions.RuntimeError: Couldn't find executable for > 'svn'" > > + the same for git > > I added ~buildbot/.bashrc to hopefully add Fink stuff > to buildbot's environment > > .hc > > > Hmm, didn't work out, and > net.sourceforge.buildbot.plist has sw/bin in
the path > too, and svn is there, so i really don't know... > > Andras > > > Ok, I was getting some builds from macosx104-i386, but then it > disappeared, donno what happened there. macosx104-powerpc > seems to be running still tho. > > > Yea the process died somehow. I have restarted it with the command: > /sw/bin/buildbot restart /Users/buildbot/macosx104-i386 > > The output had some complaints: > /sw/lib/python2.6/site-packages/twisted/persisted/sob.py:12:
> DeprecationWarning: the md5 module is deprecated; use hashlib instead > import os, md5, sys > /sw/lib/python2.6/site-packages/twisted/python/filepath.py:12:
> DeprecationWarning: the sha module is deprecated; use the hashlib > module instead > import sha > /sw/lib/python2.6/site-packages/twisted/internet/_sslverify.py:5:
> DeprecationWarning: the md5 module is deprecated; use hashlib instead > import itertools, md5 > Following twistd.log until startup finished.. > /sw/lib/python2.6/site-packages/buildbot/scripts/logwatcher.py:52:
> PotentialZombieWarning: spawnProcess called, but theSIGCHLD
handler > is not installed. This probably means you have not yet called > reactor.run, or calledreactor.run(installSignalHandler=0).
You will > probably never see this process finish, and it maybecome a
zombie > process. > env=os.environ, > Removing stale pidfile /Users/buildbot/macosx104-i386/twistd.pid > > I worked on the pd-master/master.cfg a bit, including changing > some of the names to be more consistent. I alsogot
pure-data > building from Miller's git. > > http://128.238.56.50:8010/builders/pure-data%20Linux % > 20debian-stable-i386/builds/6 > > .hc > > > Good! I saw you stared experimenting with a builder forthe
externals > too - do i understand right that at the end we will have every > external built separately? I was thinking about breaking them out to a > separate master, but then we'd need to duplicate everyslave
setup, so > finally i think they could stay in the main master, and we could have > each of their have their own "category" name, which allows for some > selection at the web page. > Also note that for the builders, you can define an array with > "slavenames:" instead of a single string "slavename", soyou
can test > the same builder on multiple slaves at the same time. Atthe
end they > have to broken down to one slave per builder, otherwisethe
diag > output is not easy to understand. > > Changing descriptionDone values to past tense like "compiled" may not > make sense when the step fails and the output goes like "compiled > failed". Also there are things like "autogen" which don't have a > proper past tense... :) > Another thing i noticed an "svn update" by itself, i think we shall > have the sources explicitly in master.cfg otherwise itwill
fail where > the slave got reset. Also you told before we wanted "clobber" (tabula > rasa) checkout not an update... > I saw the "make install", "make uninstall" steps in the output - > having these would make much sense, fyi tests can becalled
with > Test() which has some advantages over ShellCommand()like it
doesn't > make the whole build halt on failure. > BTW switching the sources to git is easy, what we have to work out is > Git polling. It's built into 0.8.1 but needs to triggered from git for > 0.7.12. And then we have this thing with the poller to explain it > which builder to start upon updates... a.k.a. the cake :) > > Andras One approach would be to use multiple PBChangeSource things and have commit hooks report to buildbot that they should build:Actually there is only one PBChangeSource needed/possible because
it's
a listener, and multiple commit hooks can communicate with it from
the
repos.
Another promising thing is loki, a web interface for easily setting up master/slaves: https://fedorahosted.org/loki/Hm. Seems a bit young to me...
I see that at the moment every factory is "under construcion" so i won't touch the config file for a while so that you can edit around. (Which makes me think about putting it in svn later...) http://128.238.56.50:8010/one_box_per_builder
Andras
I'm actually not going to have any time to look at this for a couple weeks, so please edit away.
Ok, i will. I'm not very good at build building generally, and which steps certain pd sources need, but i'll experiment. I'll bug you if something fails and i really don't know why.
About the 'externals' collection, one idea I had was to set up a different step for each library, then have a common collection of steps for easy reuse. For the Makefile-based libs the steps could be: make, make install, make dist. For autotools, we could have ./configure, make, make install. If it uses automake, then these should also be available: make uninstall, make dist.
Perhaps that would be manageable, but it seems there might be a better way.
Sounds good to me. If i set up that failed libs don't halt the whole build, it might be a good solution, or i'll think if there's a better one.
I'll still be on email, so I can contribute here and there. Yeah, I was also thinking that the libs wouldn't stop the whole build.
.hc
----------------------------------------------------------------------------
"Free software means you control what your computer does. Non-free software means someone else controls that, and to some extent controls you." - Richard M. Stallman
I'm actually not going to have any time to look at this for a couple weeks, so please edit away.
Ok, i will. I'm not very good at build building generally, and which steps certain pd sources need, but i'll experiment. I'll bug you if something fails and i really don't know why.
I'll still be on email, so I can contribute here and there. Yeah, I was also thinking that the libs wouldn't stop the whole build.
.hc
Could you help me with these? - What is the "svn update" meant to do after rsync? Right now it just fails... - Pdx "make package" fails on linux with "dpkg-deb: control directory has bad permissions 700 (must be >=0755 and <=0775)", i suppose it's easy to come around this, but i guess from the Makefile... - Git update fails with "fatal: ambiguous argument '14159': unknown revision or path not in the working tree. Use '--' to separate paths from revisions". Is it a buldbot bug (tries to use svn version for git?) or do you have another idea?
Thanks,
Andras
On Sep 20, 2010, at 3:27 PM, András Murányi wrote:
I'm actually not going to have any time to look at this for a couple weeks, so please edit away.
Ok, i will. I'm not very good at build building generally, and which steps certain pd sources need, but i'll experiment. I'll bug you if something fails and i really don't know why.
I'll still be on email, so I can contribute here and there. Yeah, I was also thinking that the libs wouldn't stop the whole build.
.hc
Could you help me with these?
- What is the "svn update" meant to do after rsync? Right now it
just fails...
The rsync server is updated 2-3 times a day, the "svn update" is meant to get any more recent changes. I fixed that so it should fail anymore.
- Pdx "make package" fails on linux with "dpkg-deb: control
directory has bad permissions 700 (must be >=0755 and <=0775)", i suppose it's easy to come around this, but i guess from the Makefile...
Yeah, this is annoying because builtbot sets the umask so that group and other get no permissions. but to build a .deb package you need to have the files at least 0755 (user-rwx, group-rx, other-rx). I'm not quite sure how to handle it. My guess is that it would make sense to set the umask on the buildbot builds to have group/other read/execute perms. I think that's umask 022.
- Git update fails with "fatal: ambiguous argument '14159': unknown
revision or path not in the working tree. Use '--' to separate paths from revisions". Is it a buldbot bug (tries to use svn version for git?) or do you have another idea?
Sounds like git and svn are fighting ;-) Sounds like the buildbot is running git with the SVN revision number: "/usr/bin/git reset --hard 14165". I guess that happens when a new SVN checkin triggers a git build. I think we need to remove the SVN tracking for a PBChangeSources style.
.hc
----------------------------------------------------------------------------
"[T]he greatest purveyor of violence in the world today [is] my own government." - Martin Luther King, Jr.
I'm actually not going to have any time to look at this for a couple
weeks, so please edit away.
Ok, i will. I'm not very good at build building generally, and which steps certain pd sources need, but i'll experiment. I'll bug you if something fails and i really don't know why.
I'll still be on email, so I can contribute here and there. Yeah, I was also thinking that the libs wouldn't stop the whole build.
.hc
Could you help me with these?
- What is the "svn update" meant to do after rsync? Right now it just
fails...
The rsync server is updated 2-3 times a day, the "svn update" is meant to get any more recent changes. I fixed that so it should fail anymore.
- Pdx "make package" fails on linux with "dpkg-deb: control directory has
bad permissions 700 (must be >=0755 and <=0775)", i suppose it's easy to come around this, but i guess from the Makefile...
Yeah, this is annoying because builtbot sets the umask so that group and other get no permissions. but to build a .deb package you need to have the files at least 0755 (user-rwx, group-rx, other-rx). I'm not quite sure how to handle it. My guess is that it would make sense to set the umask on the buildbot builds to have group/other read/execute perms. I think that's umask 022.
Seems that slaves can have their own umask set in /var/lib/buildbot/slavename/buildbot.tac Set 022 on my box, let's see if it helps. ps: cannot see because an older compile error came back: "cannot find install-sh, install.sh, or shtool"
- Git update fails with "fatal: ambiguous argument '14159': unknown
revision or path not in the working tree. Use '--' to separate paths from revisions". Is it a buldbot bug (tries to use svn version for git?) or do you have another idea?
Sounds like git and svn are fighting ;-) Sounds like the buildbot is running git with the SVN revision number: "/usr/bin/git reset --hard 14165". I guess that happens when a new SVN checkin triggers a git build. I think we need to remove the SVN tracking for a PBChangeSources style.
Done. buildbot 0.8.1 has GitPoller() so one day we may be able to set it up. Meanwhile we could take a look how this PBChangeSource() can be triggered from the sourceforge git. One way is contrib/git_buildbot.py, another a commit script calling "buildbot sendchange" and the last resort is parsing commit mails ("post-receive-email" for git) with a MaildirSource() which is to be written up. Knowing that GitPoller() already exists i'd recommend the simplest solution but which one is that? :)
Andras
On Sep 21, 2010, at 6:57 PM, András Murányi wrote:
I'm actually not going to have any time to look at this for a couple weeks, so please edit away.
Ok, i will. I'm not very good at build building generally, and which steps certain pd sources need, but i'll experiment. I'll bug you if something fails and i really don't know why.
I'll still be on email, so I can contribute here and there. Yeah, I was also thinking that the libs wouldn't stop the whole build.
.hc
Could you help me with these?
- What is the "svn update" meant to do after rsync? Right now it
just fails...
The rsync server is updated 2-3 times a day, the "svn update" is meant to get any more recent changes. I fixed that so it should fail anymore.
- Pdx "make package" fails on linux with "dpkg-deb: control
directory has bad permissions 700 (must be >=0755 and <=0775)", i suppose it's easy to come around this, but i guess from the Makefile...
Yeah, this is annoying because builtbot sets the umask so that group and other get no permissions. but to build a .deb package you need to have the files at least 0755 (user-rwx, group-rx, other-rx). I'm not quite sure how to handle it. My guess is that it would make sense to set the umask on the buildbot builds to have group/other read/execute perms. I think that's umask 022.
Seems that slaves can have their own umask set in /var/lib/buildbot/ slavename/buildbot.tac Set 022 on my box, let's see if it helps. ps: cannot see because an older compile error came back: "cannot find install-sh, install.sh, or shtool"
Arg, yes, stupid mistake, should be fixed.
- Git update fails with "fatal: ambiguous argument '14159': unknown
revision or path not in the working tree. Use '--' to separate paths from revisions". Is it a buldbot bug (tries to use svn version for git?) or do you have another idea?
Sounds like git and svn are fighting ;-) Sounds like the buildbot is running git with the SVN revision number: "/usr/bin/git reset --hard 14165". I guess that happens when a new SVN checkin triggers a git build. I think we need to remove the SVN tracking for a PBChangeSources style.
Done. buildbot 0.8.1 has GitPoller() so one day we may be able to set it up. Meanwhile we could take a look how this PBChangeSource() can be triggered from the sourceforge git. One way is contrib/ git_buildbot.py, another a commit script calling "buildbot sendchange" and the last resort is parsing commit mails ("post- receive-email" for git) with a MaildirSource() which is to be written up. Knowing that GitPoller() already exists i'd recommend the simplest solution but which one is that? :)
Andras
The pollers are nice, but we can't use them because they assume that all builds are coming from that single git source. If we want to setup multiple masters, then we could. It sounds like the thing to do is investigate the PBChangeSource stuff to see if we can have the pure- data SVN and multiple git sources
.hc
----------------------------------------------------------------------------
"We have nothing to fear from love and commitment." - New York Senator Diane Savino, trying to convince the NY Senate to pass a gay marriage bill
I'm actually not going to have any time to look at this for a couple weeks, so please edit away.
Ok, i will. I'm not very good at build building generally, and which steps certain pd sources need, but i'll experiment. I'll bug you if something fails and i really don't know why.
I'll still be on email, so I can contribute here and there. Yeah, I was also thinking that the libs wouldn't stop the whole build.
.hc
Could you help me with these?
- What is the "svn update" meant to do after rsync? Right now it just
fails...
The rsync server is updated 2-3 times a day, the "svn update" is meant to get any more recent changes. I fixed that so it should fail anymore.
- Pdx "make package" fails on linux with "dpkg-deb: control directory has
bad permissions 700 (must be >=0755 and <=0775)", i suppose it's easy to come around this, but i guess from the Makefile...
Yeah, this is annoying because builtbot sets the umask so that group and other get no permissions. but to build a .deb package you need to have the files at least 0755 (user-rwx, group-rx, other-rx). I'm not quite sure how to handle it. My guess is that it would make sense to set the umask on the buildbot builds to have group/other read/execute perms. I think that's umask 022.
Seems that slaves can have their own umask set in /var/lib/buildbot/slavename/buildbot.tac Set 022 on my box, let's see if it helps. ps: cannot see because an older compile error came back: "cannot find install-sh, install.sh, or shtool"
Arg, yes, stupid mistake, should be fixed.
and umask settings works, now it's set on all linux boxes (on macs it seems no problem)
- Git update fails with "fatal: ambiguous argument '14159': unknown
revision or path not in the working tree. Use '--' to separate paths from revisions". Is it a buldbot bug (tries to use svn version for git?) or do you have another idea?
Sounds like git and svn are fighting ;-) Sounds like the buildbot is running git with the SVN revision number: "/usr/bin/git reset --hard 14165". I guess that happens when a new SVN checkin triggers a git build. I think we need to remove the SVN tracking for a PBChangeSources style.
Done. buildbot 0.8.1 has GitPoller() so one day we may be able to set it up. Meanwhile we could take a look how this PBChangeSource() can be triggered from the sourceforge git. One way is contrib/git_buildbot.py, another a commit script calling "buildbot sendchange" and the last resort is parsing commit mails ("post-receive-email" for git) with a MaildirSource() which is to be written up. Knowing that GitPoller() already exists i'd recommend the simplest solution but which one is that? :)
Andras
The pollers are nice, but we can't use them because they assume that all builds are coming from that single git source. If we want to setup multiple masters, then we could. It sounds like the thing to do is investigate the PBChangeSource stuff to see if we can have the pure-data SVN and multiple git sources
Alrite, it's a whole new world for me... We have git sources, and it seems we'll continue having some svn sources, and is rsync a different source then? They'll have to trigger the "buildbot sendchange --plus lots of args" command on any box that has buildbot installed. There are some examples in /usr/share/buildbot/contrib/, but i think the first thing to figure out is how we gonna hook post-receive on each of these source. If we cannot hook (is it possible at all on sourceforge?) we'll have to use mails.
Andras
On Sep 14, 2010, at 4:08 PM, András Murányi wrote:
Ok 128.238.56.59 (macosx104-i386) is setup with a buildbot user with your ssh key, and buildbot is setup as launchd service: http://buildbot.net/trac/wiki/UsingLaunchd
Aww, heavy reading! How can i run the 'buildbot create-slave' command just once?
You can run it in the buildbot account using:
sudo /sw/bin/buildbot create-slave
done, up and running
Could you send me this command off-list? I want to run it on the macosx104-powerpc, which currently does not have an externally accessible IP.
The file is here, let me know if you need me to change it (probably): /Library/LaunchDaemons/net.sourceforge.buildbot.plist
It seems only this... <key>WorkingDirectory</key> <string>/Users/buildbot/slave/test</string> ...needs to have the right path which shall be /Users/buildbot/ macosx104-i386 once the slave is created.
If it cannot be run manually, we can try to fake it by copying (and modding) the files from another slave, shall we?
Ok, I edited it, and gave 'buildbot' rw perms, so you can edit if need be. Let me know when you are done so I can convert it back to being owned by root.
.hc
This is done too. At the moment buildbot fails with "exceptions.RuntimeError: Couldn't find executable for 'svn'"
Andras
Ah, ok, Mac OS X is a bit weird since it doesn't have its own package management. We use Fink on the build machines, and Fink installs into its own path: /sw. So to get lots of tools and libraries, Fink needs to be added to the relevant paths. So for the PATH for buildbot, it'll need /sw/bin. You can see in the Makefiles, the include path is set using -I/sw/include and lib path using -L/sw/lib.
.hc
----------------------------------------------------------------------------
You can't steal a gift. Bird gave the world his music, and if you can hear it, you can have it. - Dizzy Gillespie
-
András Murányi -
Hans-Christoph Steiner