wiki:RunningBuildbotWithVirtualEnv

virtualenv is a really handy Python tool that builds "sandboxes" where you can install all the Python modules you want, without affecting your system's Python. Here's how to use it to develop Buildbot.

Getting Started

First, get virtualenv installed. You'll need to install this in your system's site-packages, but it's easy. If your package manager provides a way to do this, use it instead.

sudo easy_install virtualenv

Set yourself up a working directory with a sandbox and some source

mkdir buildbot-work
cd buildbot-work
git clone git://github.com/buildbot/buildbot.git src
virtualenv sandbox

Activate the sandbox

source sandbox/bin/activate

or on Windows

sandbox\Scripts\activate

(you should see a (sandbox) prefixed to your shell prompt)

Install buildbot with the 'develop' option so that you can hack on the active code.

cd src
pip install -e master
pip install -e slave

Now install the web UI. You have two choices here, depending on whether you'll be hacking on it or not.

If you won't be hacking any Coffeescript, Jade, or CSS, you can install the web UI from the latest package on pypi, (c.f. #2822):

pip install buildbot-www

If you will be hacking on the UI, you'll need the latest versions of Node and NPM to rebuild it. The versions distributed by your operating system or packaging system are probably not new enough. See https://gist.github.com/isaacs/579814. Even with new versions, this often fails and the errors from npm, bower, and grunt are inscrutible, to put it kindly. One common issue is, on Debian systems, a missing node binary -- the Debian developers have seen fit to rename node to nodejs. Install nodejs-legacy to add a symlink with the correct name.

Once that's done,

pip install -e pkg
pip install -e www/base

You may also want www/console_view, etc. if you need other www plugins.

Install mock so you can run unit tests

pip install mock

For patch development, you may also want sphinx (to build docs) and pyflakes (for static analysis of new patches)

pip install sphinx
pip install pyflakes

Note: On windows it's necessary to install win32api as well, download from here the latest executable appropriate for your system, and then within the sandbox run:

easy_install <executable_name>

Hacking

If necessary, re-activate your sandbox (see above).

Hack away. To run unit tests:

cd src
trial buildbot.test buildslave.test

Note: On Windows, you give search path to trial

Running

To run buildbot, you'll need to go through the usual buildbot setup process. First, make sure your sandbox is active (see above). Then set up a master directory:

Master

cd sandbox
buildbot create-master master

create and edit master/master.cfg up to your liking, then, to start the buildmaster,

buildbot start $VIRTUAL_ENV/master

or, if you want to run it in the foreground,

twistd --nodaemon --no_save -y $VIRTUAL_ENV/master/buildbot.tac

Slave

If you're using the sample config:

  • host: localhost:9989
  • name will be "example-slave"
  • password is in the sample config ("pass").
    buildslave create-slave slave localhost:9989 example-slave pass
    

Then start it in the background with

buildslave start $VIRTUAL_ENV/slave

or in the foreground with

twistd --nodaemon --no_save -y $VIRTUAL_ENV/slave/buildbot.tac

www and pkg .. wtf?

The web UI requires at least the 'buildbot-www' package, which corresponds to the source code in www/base, be installed. Installing this package requires a lot of extra equipment that your typical Python hacker doesn't have (node, npm, etc.)

One option for getting this package installed is just to pip install from pypi -- this gets the most recently released version. When #2822 is finished, you'll be able to pip install the absolute latest version, rather than the latest release.

The other option, if you want to hack on the JS stuff, is to run the 'buildbot-www' package from the source tree, with pip install -e www/base. However, the package's setup.py script depends on some code in the 'buildbot-pkg' package, so you'll need to install that first. That's as easy as pip install -e pkg. The 'buildbot-pkg' package contains some code that's common to all www plugins that we don't want to copy/paste between them.

So, it's annoying to deal with all of this, but it works in the end!

Last modified 5 weeks ago Last modified on Aug 20, 2014, 12:20:34 AM