virtualenv is a really handy Python tool that builds "sandboxes" where you can install all the Python modules you want, without affecting your system's Python. Here's how to use it to develop Buildbot.
First, get virtualenv installed. You'll need to install this in your system's site-packages, but it's easy. If your package manager provides a way to do this, use it instead.
sudo easy_install virtualenv
Set yourself up a working directory with a sandbox and some source
mkdir buildbot-work cd buildbot-work virtualenv sandbox
Activate the sandbox
or on Windows
(you should see a (sandbox) prefixed to your shell prompt)
Install From Pip
pip install -U pip pip install 'buildbot[bundle]'
Install From Source
Install buildbot with the 'develop' option so that you can hack on the active code.
git clone git://github.com/buildbot/buildbot.git src cd src pip install -e master pip install -e worker
Now install the web UI. You have two choices here, depending on whether you'll be hacking on it or not.
I'm NOT a Web Developer
If you won't be hacking any Coffeescript, Jade, or CSS, you can install the web UI from the latest build:
I'm a Web Developer
If you will be hacking on the UI, you'll need the latest versions of Node and NPM to rebuild it. The versions distributed by your operating system or packaging system are probably not new enough. See https://gist.github.com/isaacs/579814. Even with new versions, this often fails and the errors from npm, bower, and grunt are inscrutible, to put it kindly. One common issue is, on Debian systems, a missing node binary -- the Debian developers have seen fit to rename node to nodejs. Install nodejs-legacy to add a symlink with the correct name.
Once that's done, following command will build the web ui and plugins
The pkg directory contains some common code that must be installed before the www plugins. Source code is located in www
Install mock so you can run unit tests
pip install mock
For patch development, you may also want sphinx (to build docs) and pyflakes (for static analysis of new patches)
pip install sphinx pip install pyflakes
Note: On windows it's necessary to install win32api as well, download from here the latest executable appropriate for your system, and then within the sandbox run:
If necessary, re-activate your sandbox (see above).
Hack away. To run unit tests:
cd src trial buildbot.test buildbot_worker.test
Note: On Windows, you give search path to trial
To run buildbot, you'll need to go through the usual buildbot setup process. First, make sure your sandbox is active (see above). Then set up a master directory:
cd sandbox buildbot create-master master
create and edit master/master.cfg up to your liking, then, to start the buildmaster,
buildbot start $VIRTUAL_ENV/master
or, if you want to run it in the foreground,
twistd --nodaemon --no_save -y $VIRTUAL_ENV/master/buildbot.tac
If you're using the sample config:
- host: localhost:9989
- name will be "example-worker"
- password is in the sample config ("pass").
buildbot-worker create-worker worker localhost:9989 example-worker pass
Then start it in the background with
buildworker start $VIRTUAL_ENV/worker
or in the foreground with
twistd --nodaemon --no_save -y $VIRTUAL_ENV/worker/buildbot.tac
www and pkg .. wtf?
The web UI requires at least the 'buildbot-www' package, which corresponds to the source code in www/base, be installed. Installing this package requires a lot of extra equipment that your typical Python hacker doesn't have (node, npm, etc.)
One option for getting this package installed is just to pip install from ftp://ftp.buildbot.net/pub/latest/ or http://ftp.buildbot.net/pub/latest/ -- this gets the most recently committed version, placed there by the metabuildbot. The other option, if you want to hack on the JS stuff, is to run the 'buildbot-www' package from the source tree, with pip install -e www/base. However, the package's setup.py script depends on some code in the 'buildbot-pkg' package, so you'll need to install that first. That's as easy as pip install -e pkg. The 'buildbot-pkg' package contains some code that's common to all www plugins that we don't want to copy/paste between them.
So, it's annoying to deal with all of this, but it works in the end!