wiki:FAQ

Common Pitfalls and Gotchas - Frequently Asked Questions

When I try using the ShellCommand build step I get "OSError: [Errno 2] No such file or directory" even though the file, or directory seems to be there.

Buildbot sends commands to the shell as strings. When you specify the command argument to ShellCommand you should pass a list of strings for each argument in that command rather than the entire command as a string in a single-element list.

Dont do this: factory.addStep(ShellCommand(command=['sh /path/to/script.sh']))

Do this instead: factory.addStep(ShellCommand(command=['sh', '/path/to/script.sh']))

Can I use a single buildmaster for multiple projects?

For simple cases, this is now quite possible. A single buildmaster can handle an arbitrary number of distinct projects, even running the same builders on different projects. The project and repository attributes are used to separate the projects and indicate which version-control repository contains the code for each project.

However, a project that packages several independently-developed utilities into a single binary for users to download is more difficult, as Buildbot does not have a means to represent the multiple revisions from multiple repositories that must be used to make up the single binary. For simple cases (e.g,. always building the latest version), this can be made to work, but more complex cases are still a work in progress.

Consult the mailing list archives for discussion of various implementations of multi-project buildmasters.

Why don't wildcards work in my ShellCommand?

Probably because your worker is running a Unix variant and you wrote your command as a list (the recommended way):

factory.addStep(ShellCommand(
    command=["rm", "*.old"],
    ...))

The advantage of writing your command as a list is that it is executed directly by BuildBot, without a shell getting in the way. That means you won't have problems with quoting or whitespace in filenames. But since there is no shell involved, it also means that wildcards will not be expanded: BuildBot simply passes the string *.old to rm, which complains because there is no such file.

The solution is to specify your command as a string, so it will be run by /bin/sh:

factory.addStep(ShellCommand(
    command="rm *.old",
    ...))

However, this can introduce all the classic shell quoting nightmares.

Alternately, you can get BuildBot to explicitly run the command with /bin/sh:

factory.addStep(ShellCommand(
    command=["/bin/sh", "-c", "rm *.old"],
    ...))

Why don't redirects/pipes/etc work in my ShellCommand?

For the exact same reason that wildcards don't work. See the previous question.

Why isn't buildbot designed for my build process, since it's a very common process?

Most people feel that their build proces is "normal", "obvious", or "common". It's not. No two buildbot users have the same build process, nor do they organize their code in the same way. You can believe this now, or watch the mailing list and IRC channel for a week or so and believe it later.

My Mac worker can't resolve domain names

Macs use "security contexts", so if you start a worker via an SSH connection, then once that connection disappears, the context does, too. See UsingLaunchd or UsingStartupItemContext for workarounds.

I'm getting errors about overly-long filenames

This is surprisingly common, especially on Windows and, to a lesser extent, Mac workers, when you have tests that create lots of nested directories. There's not a lot that Buildbot can do, but here are some ideas:

  • set up your worker root directories in a short top-level directory, e.g.,
    buildbot-worker create-worker /B master:9989 botname botpass
    
  • use abbreviated builddirs in your builder configurations:
    c['builders'] = [ 
           BuilderConfig(
               name="my really long builder name",
               builddir="mrlbn",
               ...
           ) ]
    

With both of these in place, Buildbot will only contribute /B/mrlbn to the pathnames - the other 247 characters will be up to the developers to remove.

How can I trigger build from SVN post-commit hook?

Assuming you have buildbot master running on buildbot.example.com with PBListener on port 9989 your post-commit hook should look like this:

#!/bin/sh
# Subversion post-commit hook.

REV=$2
REPOS=$1

/usr/share/buildbot/contrib/svn_buildbot.py 
    --repository "$REPOS" 
    --revision "$REV" 
    --bbserver buildbot.example.com 
    --bbport 9989

I'm seeing "LogFileScanner has no attribute _remainingData"

This is an incompatibility in versions Buildbot-0.8.2 and earlier when running against Twisted 10.2.0 - see #1697. A few solutions are available:

  • downgrade Twisted (e.g., pip install Twisted==10.1.0)
  • upgrade Buildbot to 0.8.3 or later
  • apply the fix

The GitPoller is not working!

Buildbot-0.8.3 shipped with a badly broken gitpoller (oops!). The poller is fixed in Buildbot-0.8.3p1, so please use that version or a later version instead.

My Git steps on Windows aren't working!

If you're using an old version of MSys Git, you may see problems where the exit status of a git command is incorrectly passed back to Buildbot, causing Source steps to fail. Upgrade to at least 1.7.3.1.

The Console view only displays white boxes

This happens with version-control systems that do not use integers for revisions (that is, most everything but SVN). You'll need to add order_console_by_time=True to your WebStatus constructor in master.cfg.

I'd like to run arbitrary Python code on the worker

We've historically resisted putting "execute this arbitrary Python" support into Buildbot, for two reasons:

  1. it's more complex than it sounds: It boils down to sending a string on which the remote system calls exec(), but with blocking and potentially damaging ramifications for the running worker instance. Steer clear of thoughts of sending compiled Python bytecode across the wire, too.
  2. it's generally the wrong solution to the problem you're trying to solve: If the functionality is so complex that it cannot be represented in a few lines of shell script, then most likely that functionality will need to be revision-controlled and tested outside of Buildbot. Which means that it should be in a Python script in version-control somewhere, and Buildbot should be running the code from there.

I'm getting import errors: cannot import name deque

  File "/usr/local/lib/python2.6/dist-packages/buildbot-0.8.1rc3_1536_gc2ef324-py2.6.egg/buildbot/util/lru.py", line 20, in <module>
    from collections import deque
exceptions.ImportError: cannot import name deque

This is because there is a 'collections.pyc' floating around in your Buildbot install, after upgrading from an earlier version. Seek and destroy it.

How do I "reset" my master to delete existing build state?

The easiest way to do this is to save your configuration, delete the master directory, and re-run buildbot create-master, then re-install your configuration.

My custom step seems to be executing on the master!

Yep. Buildbot doesn't magically execute Python written on the master on the worker, or vice versa. All it does is send instructions to the worker, and often those instructions involve invoking some other program via the shell.

Last modified 6 months ago Last modified on Jan 4, 2017, 2:15:11 AM