virtualenv is a really handy Python tool that builds "sandboxes" where you can install all the Python modules you want, without affecting your system's Python. Here's how to use it to develop Buildbot.
First, get virtualenv installed. You'll need to install this in your system's site-packages, but it's easy. If your package manager provides a way to do this, use it instead.
sudo easy_install virtualenv
Set yourself up a working directory with a sandbox and some source
mkdir buildbot-work cd buildbot-work git clone git://github.com/buildbot/buildbot.git src virtualenv sandbox
Activate the sandbox
or on Windows
(you should see a (sandbox) prefixed to your shell prompt)
Install buildbot with the 'develop' option so that you can hack on the active code.
cd src pip install -e master pip install -e slave
Now install the web UI. You have two choices here, depending on whether you'll be hacking on it or not.
If you won't be hacking any Coffeescript, Jade, or CSS, you can install the web UI from the latest package on pypi, (c.f. #2822):
pip install buildbot-www
If you will be hacking on the UI, you'll need the latest versions of Node and NPM to rebuild it. The versions distributed by your operating system or packaging system are probably not new enough. See https://gist.github.com/isaacs/579814. Even with new versions, this often fails and the errors from npm, bower, and grunt are inscrutible, to put it kindly. One common issue is, on Debian systems, a missing node binary -- the Debian developers have seen fit to rename node to nodejs. Install nodejs-legacy to add a symlink with the correct name.
Once that's done,
pip install -e pkg pip install -e www/base
You may also want www/console_view, etc. if you need other www plugins.
Install mock so you can run unit tests
pip install mock
For patch development, you may also want sphinx (to build docs) and pyflakes (for static analysis of new patches)
pip install sphinx pip install pyflakes
Note: On windows it's necessary to install win32api as well, download from here the latest executable appropriate for your system, and then within the sandbox run:
If necessary, re-activate your sandbox (see above).
Hack away. To run unit tests:
cd src trial buildbot.test buildslave.test
Note: On Windows, you give search path to trial
To run buildbot, you'll need to go through the usual buildbot setup process. First, make sure your sandbox is active (see above). Then set up a master directory:
cd sandbox buildbot create-master master
create and edit master/master.cfg up to your liking, then, to start the buildmaster,
buildbot start $VIRTUAL_ENV/master
or, if you want to run it in the foreground,
twistd --nodaemon --no_save -y $VIRTUAL_ENV/master/buildbot.tac
If you're using the sample config:
- host: localhost:9989
- name will be "example-slave"
- password is in the sample config ("pass").
buildslave create-slave slave localhost:9989 example-slave pass
Then start it in the background with
buildslave start $VIRTUAL_ENV/slave
or in the foreground with
twistd --nodaemon --no_save -y $VIRTUAL_ENV/slave/buildbot.tac
www and pkg .. wtf?
The web UI requires at least the 'buildbot-www' package, which corresponds to the source code in www/base, be installed. Installing this package requires a lot of extra equipment that your typical Python hacker doesn't have (node, npm, etc.)
One option for getting this package installed is just to pip install from pypi -- this gets the most recently released version. When #2822 is finished, you'll be able to pip install the absolute latest version, rather than the latest release.
The other option, if you want to hack on the JS stuff, is to run the 'buildbot-www' package from the source tree, with pip install -e www/base. However, the package's setup.py script depends on some code in the 'buildbot-pkg' package, so you'll need to install that first. That's as easy as pip install -e pkg. The 'buildbot-pkg' package contains some code that's common to all www plugins that we don't want to copy/paste between them.
So, it's annoying to deal with all of this, but it works in the end!