The Buildbot project is seeking Windows experts to coordinate Windows compatibility. Contact dustin@…
Buildbot on Windows - Installation Instructions
Buildbot runs on Windows, as both a worker and a master. The master can run on any convenient platform.
Prerequisites for a worker and for a master
- Python 2.x. Use the latest 2.x version of Python.
- PyWin32. Match the version with the Python you installed.
- Twisted. Match the version with the Python you installed.
- setuptools. Match the version with the Python you installed.
- Use easy_install to install Zope.Interface: Open a Windows command prompt (cmd.exe) and run "C:\Python27\scripts\easy_install zope.interface".
- Buildbot, of course. You need 0.8.2 or later. Use C:\Python27\scripts\easy_install whenever you see easy_install in the generic instructions.
Additional Prerequisites for a master
- Jinja2. This will be installed using setuptools and then easy_installing it via "C:\Python27\scripts\easy_install Jinja2".
- PyOpenSSL. This will be installed using setuptools and then easy_installing it via "C:\Python27\scripts\easy_install PyOpenSSL".
For the remainder of this document we will assume Python was installed in "C:\Python27". Please adjust accordingly if you install to a different path.
Install Python, pywin32, and Twisted for all users. On Windows Server 2003 it was found that this MUST be done while logged in as the actual "Administrator" account, NOT as a regular user with Administrator privileges, because of the way filesystem permissions are inherited during egg installation. The Buildbot service itself need not be, and probably should not be, a privileged account (see below). When running python setup.py, be sure to use the full path to the version of python you expect to run Buildbot (It is ok to have multiple versions of python installed). The pywin32 and Twisted installers will be matched for the version of python that will be running Buildbot.
Install Buildbot master with
C:\Python27\python setup.py install
The following steps were successfully used on Windows Server 2003 SP2 (32 bit) using python 2.7 for buildbot 0.8.3p1. They should work for later versions of Buildbot. Note the use of an absolute path to the Python executable during installation; this is required if you have multiple side-by-side installations of Python
- log in as Administrator directly (ctrl-alt-delete from the login screen) Doing this as any other user, even users with Administrator privileges, will cause the eggs to be installed with incorrect permissions and a non-privileged Buildbot service will fail to launch.
- Download the above packages via Firefox, for example into "C:\Documents and Settings\Administrator\Desktop". IE will try to turn the egg file into a zip file.
- Run a cmd prompt as Administrator
- Type cd Desktop
- Type .python-2.7.1.msi Click Run, Leave default Install for All Users, click Next, leave default Python27 destination, click Next, click Next, click Finish.
- Type .pywin32-214.win32-py2.7.exe Click Next, Leave default of C:\Python27 click Next, click Next, click Finish.
- Type .Twisted-10.2.0.winxp32-py2.7.exe Click Next, Leave default of C:\Python27, click Next, click Next, click Finish.
- Type .setuptools-0.6c11.win32-py2.7.exe Click Next, Leave default of C:\Python27, click Next, click Next, click Finish.
- Type C:\Python27\scripts\easy_install.exe zope.interface-3.6.1-py2.7-win32.egg
- If you are installing a master, install Jinja2 using the easy_install method from the Jinja2 website.
- If you are installing a master, install the correct version of pyOpenSSL from the 2.7 egg, from here https://launchpad.net/pyopenssl . This is necessary for SSL email to work (build failure emails).
- If you are installing a master, unzip buildbot-0.8.5.zip in this folder. type cd buildbot-0.8.5uildbot-0.8.5, type c:\Python27\python.exe setup.py install
- If you are installing a worker, unzip buildbot-slave-0.8.5.zip in this folder. type cd buildbot-slave-0.8.5, type C:\Python27\python.exe setup.py install
Test Your Installation
Verify the worker installation. For example:
C:\Documents and Settings\Administrator\Desktop>C:\Python27\scripts\buildslave --version Buildslave version: 0.8.3 Twisted version: 10.2.0
Create a User
The rest of this document assumes you are running as a non-privileged user. We recommend creating one specifically for your worker.
Basic setup under windows is the same as under other operating systems, except that BuildBot currently cannot manage masters or workers whose paths have spaces in them. If you absolutely must use a path with spaces, you can map that path to a drive letter and refer to it that way. Run buildslave create-slave <options> to create your slave directory. Edit the info files in the worker directory as appropriate. Then start the worker process. Check twistd.log for any error messages.
You can call it quits and be done at this point. Any time the command prompt buildslave is running under closes, your worker will shutdown -- this includes reboots for windows update! Thus it is highly desirable to install buildbot as a windows service. When installed as a service, Windows will automatically restart your worker after a reboot.
You'll need to grant your non-privileged user the ability to run services. To do this, run secpol.msc as an Administrator (the author found the easiest way to do this was to right-click a command line prompt and "run as administrator", and then enter secpol.msc in the command line). Then:
- Select the "Local Policies" folder
- Select the "User Rights Assignment" folder
- Double click "Log on as a service"
- Use "Add User or Group..." to add your non-privileged user here.
If you don't already have one open, open up a command shell running as administrator. Then use a builtin buildbot script to install a service for buildbot:
C:\Python27\python.exe C:\Python27\Scripts\buildbot_service.py --user YOURDOMAIN\theusername --password thepassword --startup auto install
(except all as one line, of course!) Note the "YOURDOMAIN" part is required. For most setups, you can replace "YOURDOMAIN" with ".", making the full user name ".\theusername" (where theusername is the username of the non-privileged user, and thepassword is that user's login password). Ignore any messages about "Failed to register with the windows firewall".
The next bit of setup involves adding permissions on a certain registry key. However, the registry key does not yet exist because the service has never been started. Under Administrative Tools find Services and run it as an administrator. Then start the "Buildbot" service. Windows will immediately complain that the service shutdown immediately; that's fine.
In your administrator command prompt, run "regedit". Use it to navigate to "HKEY_LOCAL_MACHINE\System\CurrentControlSet\services"; in that long list of services you will find a "Buildbot" entry (i.e. folder). Right click the Buildbot folder and select permissions. In the dialog that comes up, add a new user to the list, and then grant that user "Full Control".
Now try again to start the service. Windows will again immediately complain that the service shutdown immediately; that's fine. This step is required in order to create the "Parameters" key we'll use in the next step.
Under that "Buildbot" services in the registry you will find a "Parameters" key (you may need to hit F5 to refresh so you can see it). There are currently no values under this key. You will need to *create* a 'String Value' under "Parameters" which is named "directories". Set the value of "directories" to be the full path to your master and/or slave's configuration directory. This is the same directory where buildbot.tac and twistd.log live. For example, a master and worker could be "C:\Buildbot\master;C:\Buildbot\slave". Alternatively, you can use PowerShell?:
New-Item -path Registry::HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\BuildBot New-Item -path Registry::HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\BuildBot\Parameters New-ItemProperty -path Registry::HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\BuildBot\Parameters -Name directories -PropertyType String -Value <BUILDBOT DIRECTORY>
You will now need to setup a buildbot master and/or workers into the directory you specified. See Build Bot Tutorial: First Run for more information.
Finally, go back to your "Services" running as administrator and right-click "Buildbot" to start your service -- or just reboot.
Note: any environment variables that need to be set on the worker machine for your bot's build commands to work have to be set in the global System Environment settings or (better) in the unprivileged user's User Environment settings.
In a virtualenv
If you choose to run your service in a virtualenv, there are a few further issues you'll need to take care of.
First, PyWin32 won't install directly into a virtualenv, so you'll have to install it globally.
The service runs an executable installed by PyWin32 to "C:\Python26\Lib\site-packages\win32\PythonService.exe". When you're doing the regedit steps above you'll see that path in the ImagePath key. PythonService.exe looks for python.exe in a folder relative to where it is installed, so you'll need to create some symlinks (or on pre-Vista systems, copies). Change that ImagePath to point into your virtualenv, e.g. C:\Users\buildslave\venv\Lib\site-packages\win32\PythonService.exe. Then symlink C:\Python26\Lib\site-packages\win32 into the corresponding place in your virtualenv. Finally, make a symlink for python, which is in your virtualenv's Scripts directory, in the root of the virtualenv.
Next, you'll want to update the service's environment so that the virtualenv is in the PATH. This article shows how to do that. Note: any environment variables set here will not propagate to the build commands issued by the bot. Those should be set in the User Environment of the unprivileged user as described above.
Once you have mastered the basics of setting up a worker as a Win32 service, you might want to set up more workers on the same system. Do not go to the trouble of cloning your BuildBot service - merely add your other worker configurations to the "directories" string value, separated by semicolons. For example:
directories REG_SZ C:\BuildBotProj1;C:\BuildBotProj2;C:\BuildBotProj3
Consider using vms to manage your workers. Create a clean base installation and clone it for each new build.
Restart the service and your new workers will connect.
Disabling JIT dialogs on the worker
If you are running tests that occasionally fail with exceptions on the worker, you will want to disable the Just-In-Time (JIT) debugger that is configured with Visual Studio. When an unhandled exception occurs, the default JIT configuration will launch a dialog that will wait for user confirmation, effectively halting your build.
Follow these steps to disable the JIT:
- Run regedit (as Administrator)
- DELETE HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug\Debugger
- DELETE HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\DbgManagedDebugger
- SET HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\DbgJITDebugLaunchSetting to 0x1 (right click value, select Modify, hexadecimal, 1), ok
- If you are running a 64 bit version of windows DELETE HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\AeDebugDebugger
- If you are running a 64 bit version of windows DELETE HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\.NETFramework\DbgManagedDebugger
These settings are clarified at the following two sites. It should be noted that the registry changes from BOTH sites must be done.
Other Registry Settings
You will probably want to disable the Windows Error Reporting facility on Windows 7. This will prevent the annoying dialog box of "Report this error". Under "HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\Windows Error Reporting" add a REG_DWORD of "Disabled" and set it to "1"
If you have any issues with the service installation or buildbot-as-a-service startup, you can see the actual problem using Windows' "Event Viewer". It's under "Windows Logs" and then "Applications"; look for problems of type "Error" with a source of "Buildbot". Please report any such errors when asking for help on the mailing list.
The buildbot master.cfg needs the worker information; the master will ignore a worker it doesn't know about.
If the buildbot service fails to start, but the script runs for the same user when logged in, it may be due to permissions that were in effect when the buildbot package and prerequisites were installed. See "Prerequisites" above.
Visual Studio build configs such as "Win32|Any CPU" will need to be escaped thus: "Win32^|Any CPU". Consider saving yourself development time by using python instead of bat and cmd for worker scripts.