On Nov 29, 2016, at 12:46 AM, pd-list-request@lists.iem.at wrote:
Obiously, I would like to set up a Mac build machine as well. What do I need to install on a clean system via command line?
Install the build system via:
xcode-select —install
You don’t need to download Xcode from the app store for commandline builds.
If you need libraries, I’d recommend using Homebrew: http://brew.sh http://brew.sh/
Dan Wilcox @danomatika https://twitter.com/danomatika danomatika.com http://danomatika.com/ robotcowboy.com http://robotcowboy.com/
[:travis-ci propaganda:]
On 2016-11-29 01:09, Dan Wilcox wrote:
On Nov 29, 2016, at 12:46 AM, pd-list-request@lists.iem.at wrote:
Obiously, I would like to set up a Mac build machine as well. What do I need to install on a clean system via command line?
Install the build system via:
xcode-select —install
just in case this is not obvious: travis-ci already features OSX builders, and being a continuous integration system, these builders already come with a compiler. so - in the case of travis-ci - there is no need to manually install xcode.
If you need libraries, I’d recommend using Homebrew: http://brew.sh
i'd second that, but bear in mind that in this means that you will actually *build* the libraries (most of the times). in a CI (like travis-ci) this can lead to timeout problems: whenever a build is triggered, it starts from a pristine image. so all the dependencies need to be rebuilt, whenever a build of the main application(/library/external) is triggered. with a complex system (like Gem), i keep getting failed builds, because building the dependencies leaves little time to actually build Gem. however, you could download arbitrary (prebuilt) binaries, rather than build everything from scratch.
fgmasr IOhannes
Before PdCon16~ I didn't even know about Travis. When IOhannes explained that it is an online continuous integration tool, my naive idea was 'can we use this to build our Pd libraries on 'all' platforms and route the builds back to us?'
Many of us don't have 'all' platforms available at their home or work place and we sorely miss the Pd-extended build farm with its nightly test builds. It seems useful to share knowledge and experience about an existing infrastructure like Travis.
From https://docs.travis-ci.com/user/customizing-the-build/, I
understand that Travis CI is also a community build farm (but obviously bigger and more professional than we could organize in Pd community):
'Please take into account that Travis CI is an open source service and we rely on worker boxes provided by the community. So please only specify as big a matrix as you actually need.'
There's an info page for 'complete beginners': https://docs.travis-ci.com/user/for-beginners. A single page, is it so easy? No, you really have to click that menu button in the right top corner and read through a few dozen documentation pages to get an idea what Travis can do, and how. All the info is there presented in a user-friendly way. As we go and learn, we could make a Pd-related summary or addition to this info. Yes I'm looking forward to Thomas Mayer's 'Christmas / Chanukka gift to the community'!
If you build with Travis for a specific platform because you don't have access to that machine or OS yourself, how can you test the build products? Automated tests might be run on the Travis virtual machine if you install Pd there.
But I'm also thinking of the following (not specifically related to Travis): what if we would have a 'test section' in deken? Basically meaning an optional 'test' tag in deken package names, and a filter in the deken search plugin to show or hide test versions? Then Pd community could more easily help with testing.
Katja
On Tue, Nov 29, 2016 at 10:09 AM, IOhannes m zmoelnig zmoelnig@iem.at wrote:
[:travis-ci propaganda:]
On 2016-11-29 01:09, Dan Wilcox wrote:
On Nov 29, 2016, at 12:46 AM, pd-list-request@lists.iem.at wrote:
Obiously, I would like to set up a Mac build machine as well. What do I need to install on a clean system via command line?
Install the build system via:
xcode-select —install
just in case this is not obvious: travis-ci already features OSX builders, and being a continuous integration system, these builders already come with a compiler. so - in the case of travis-ci - there is no need to manually install xcode.
If you need libraries, I’d recommend using Homebrew: http://brew.sh
i'd second that, but bear in mind that in this means that you will actually *build* the libraries (most of the times). in a CI (like travis-ci) this can lead to timeout problems: whenever a build is triggered, it starts from a pristine image. so all the dependencies need to be rebuilt, whenever a build of the main application(/library/external) is triggered. with a complex system (like Gem), i keep getting failed builds, because building the dependencies leaves little time to actually build Gem. however, you could download arbitrary (prebuilt) binaries, rather than build everything from scratch.
fgmasr IOhannes
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
Many of us don't have 'all' platforms available at their home or work
place and we sorely miss the Pd-extended build farm with its nightly test builds. It seems useful to share knowledge and experience about an existing infrastructure like Travis.
One option is "Giblab CI" which comes with Gitlab. Essentially you define the build process
with a yml file then set up "runners" on whatever machine(s) you want. The runners can be
on bare metal or a number of container environments-- docker, virtualbox, etc.
Here's a (still rather crude) example:https://git.purrdata.net/jwilkes/purr-data/commit/326f08b8d6765482068c87e251...
The nice thing about it is that each build machine just needs a single gitlab-runner instance running
on it. That process just polls the repo waiting for relevant commits, fetches (or clones) the repo, builds,
and optionally uploads artifacts.
In my case I'm using the virtualbox backend, so on each commit it clones a virtualbox snapshot,
fetches the repo, builds, tests, and then uploads the resulting *.deb package.
The OSX box always fails because I'm trying to build inside a virtualbox OSX guest running under OSX,
and for some reason I cannot get OSX to ssh into an OSX guest. Also for Windows you'd have to, well,
have Windows. :)
But even without Windows and OSX, having the CI speeds up the dev process by an order of magnitude.
Best,
Jonathan
On Tue, Nov 29, 2016 at 5:17 PM, Jonathan Wilkes jancsika@yahoo.com wrote:
Many of us don't have 'all' platforms available at their home or work
place and we sorely miss the Pd-extended build farm with its nightly test builds. It seems useful to share knowledge and experience about an existing infrastructure like Travis.
One option is "Giblab CI" which comes with Gitlab. Essentially you define the build process with a yml file then set up "runners" on whatever machine(s) you want. The runners can be on bare metal or a number of container environments-- docker, virtualbox, etc.
Here's a (still rather crude) example: https://git.purrdata.net/jwilkes/purr-data/commit/326f08b8d6765482068c87e251...
The nice thing about it is that each build machine just needs a single
gitlab-runnerinstance running on it. That process just polls the repo waiting for relevant commits, fetches (or clones) the repo, builds, and optionally uploads artifacts.In my case I'm using the virtualbox backend, so on each commit it clones a virtualbox snapshot, fetches the repo, builds, tests, and then uploads the resulting *.deb package.
The OSX box always fails because I'm trying to build inside a virtualbox OSX guest running under OSX, and for some reason I cannot get OSX to ssh into an OSX guest. Also for Windows you'd have to, well, have Windows. :)
But even without Windows and OSX, having the CI speeds up the dev process by an order of magnitude.
It seems that with Gitlab CI you can configure 'runners' on your own hardware for the project(s) that use it, right? So you can install dependencies and don't need to start with a clean slate for every build? That has several pro's and cons as compared with Travis. More DIY, but still with a standardized API.
Katja
Best, Jonathan
It seems that with Gitlab CI you can configure 'runners' on your own
hardware for the project(s) that use it, right? Yes. I could install gitlab-runner on my rpi2, get the "token" from gitlab, and have it sit there waiting to build. In that case, however, I'd essentially be letting remote run arbitrary shell commands on the machine.
So you can installdependencies and don't need to start with a clean slate for every
build? That has several pro's and cons as compared with Travis. More DIY, but still with a standardized API. Seems most people are using the docker runner with gitlab ci. I'm not sure how that could possibly work for OSX/Windows builds, so I use virtualbox. As for "clean slate", I guess that means different things. I'm currently cloning VM snapshots with *.deb dependencies installed because it saves time. Gitlab CI also allows you to save the previously cloned VM and use it for subsequent builds. But I don't trust gitlab, virtual, or gnu make to get any of that right.
Katja
Best, Jonathan
It seems that with Gitlab CI you can configure 'runners' on your own
hardware for the project(s) that use it, right? So you can install dependencies and don't need to start with a clean slate for every build? That has several pro's and cons as compared with Travis. More DIY, but still with a standardized API.
One other feature-- you can have multiple runners for the same target. So I could have rpi drones in each timezone running gitlab-runner, and if you shoot one of them down gitlab ci will pick one of the other 23 for the next build. It's completely and utter world domination, really. -Jonathan
On Tue, Nov 29, 2016 at 12:07 PM, katja katjavetter@gmail.com wrote:
[...]
From https://docs.travis-ci.com/user/customizing-the-build/, I understand that Travis CI is also a community build farm (but obviously bigger and more professional than we could organize in Pd community):
'Please take into account that Travis CI is an open source service and we rely on worker boxes provided by the community. So please only specify as big a matrix as you actually need.'
[...]
Oops, 'community' has a different meaning here than I imagined... Travis virtual machines run on Google Compute Engine:
https://blog.travis-ci.com/2015-11-27-moving-to-a-more-elastic-future
Probably paid by private / commercial Travis users ($129 / month). Same business model as Github after all.
Katja
Hello,
On 29.11.2016 22:33, katja wrote:
On Tue, Nov 29, 2016 at 12:07 PM, katja katjavetter@gmail.com wrote:
[...]
From https://docs.travis-ci.com/user/customizing-the-build/, I understand that Travis CI is also a community build farm (but obviously bigger and more professional than we could organize in Pd community):
'Please take into account that Travis CI is an open source service and we rely on worker boxes provided by the community. So please only specify as big a matrix as you actually need.'
[...]
Oops, 'community' has a different meaning here than I imagined... Travis virtual machines run on Google Compute Engine:
https://blog.travis-ci.com/2015-11-27-moving-to-a-more-elastic-future
Probably paid by private / commercial Travis users ($129 / month). Same business model as Github after all.
Yes, and it makes sense: Leasing a computer from a server farm is really cheap now ("There is no cloud, just other people's computers"[https://fsfe.org/contribute/spreadtheword.en.html#nocloud]), and many FLOSS programmers are working as paid programmers as well, and can then spread the knowledge of these services. The "worker boxes" are the VM images.
Anyway: I have got Travis CI working for building PuREST JSON for Linux amd64 and i386, OS X, and Windows builds.
Windows builds use MXE on Linux amd64 bit with cross compiling, Linux i386 uses a chroot in Linux amd64 (Yes, I know, but http://stackoverflow.com/questions/40824438/compiling-32-bit-binaries-on-tra...), and OS X uses homebrew for installing dependencies.
The packages are then uploaded to my owncloud instance and are available from https://ssl-account.com/cloud.residuum.org/index.php/s/380C60JAabnO7jk
The basic outline: Travis CI is configured with these setups in https://github.com/residuum/PuRestJson/blob/master/.travis.yml
I have split the scripts into four parts: Part 1 (*_before_install.sh) installs the dependencies, and serves as kind-of documentation for setting up the development system (Will add it to the wiki).
Part 2 (*_script.sh) is the real build script, mostly just make, sometimes with parameters specifying that it is a build for a different machine.
Part 3 (*_pack.sh) generates the zip or tar.gz package for upload later. I will add compilation of the manual from the Github wiki to this step as well, as this is what I now do before each release.
Part 4 (upload.sh) uploads the resulting files from part 3 to my Owncloud instance with cURL.
I think, most of the things can be adapted for other packages as well, if necessary, especially for externals using pd-lib-builder like PuREST JSON.
Hth, Thomas
On 01.12.2016 00:57, Thomas Mayer wrote:
The packages are then uploaded to my owncloud instance and are available from https://ssl-account.com/cloud.residuum.org/index.php/s/380C60JAabnO7jk
OS X builds on Travis are pretty much behind since yesterday, so give it several hours time until these appear.
Thanks, Thomas
Hi Thomas,
Travis builds for W32 seems to work.
I tested on Win8.1 and in a Win10 VM running on Virtualbox.
"purest_json/examples/purest-json-test.pd" is not working but this is surely cuz "A CouchDB running on the same computer is implied "
Salutti, Lucarda.
Mensaje telepatico asistido por maquinas.
From: Pd-list pd-list-bounces@lists.iem.at on behalf of Thomas Mayer thomas@residuum.org Sent: Thursday, December 1, 2016 12:15 AM To: pd-list@lists.iem.at Subject: Re: [PD] Continuous Integration for Externals
On 01.12.2016 00:57, Thomas Mayer wrote:
The packages are then uploaded to my owncloud instance and are available from https://ssl-account.com/cloud.residuum.org/index.php/s/380C60JAabnO7jk
OS X builds on Travis are pretty much behind since yesterday, so give it several hours time until these appear.
Thanks, Thomas -- "It seems to me, Golan, that the advance of civilization is nothing but an exercise in the limiting of privacy." (Janov Pelorat in: Isaac Asimov - Foundation's Edge) http://www.residuum.org/
Pd-list@lists.iem.at mailing list UNSUBSCRIBE and account-management -> https://lists.puredata.info/listinfo/pd-list
-
Dan Wilcox -
IOhannes m zmoelnig -
Jonathan Wilkes -
katja -
Lucas Cordiviola -
Thomas Mayer