![[Buildbot Logo]](/chrome/site/header-text-transparent.png)
Table of Contents
If you're interested in working on any of these ideas, or have ideas of your own that you'd like to work on, get in touch! See the links on Development, or email the project maintainer directly - dustin@….
GSoC 2013
The projects on this page are, of course, open to anyone who wants to contribute, but the page is designed specifically for potential Google Summer of Code students. Keep in mind that we have three goals for GSoC:
- We gain new contributors to Buildbot
- You learn about open-source development by doing it
- Users get amazing new Buildbot features
We'd like to achieve all three goals, but the world is messy, and predictions are hard. If you finish the summer having learned a lot and become a lifelong contributor to Buildbot, but without wrapping up a user-visible feature, that's still OK.
Applications should follow the form given in GsocApplicationTemplate.
Open Ideas
| Ticket | Summary | Owner |
|---|---|---|
| #896 | Replace try with a client for the force scheduler | |
| Description |
We have a "try" client defined which looks for a patch in a local repository on a developer's system, patches it up, and submits it to Buildbot. It's very ad-hoc, isn't very configurable, and hasn't gotten a lot of love. Now that we have a force scheduler, the better solution is for the try client to talk to a force scheduler, preferably using HTTP. The client should also capture and send e-mail address of the submitter so that notifications can be sent. This allows lots of flexibility on the master side: users can set up as many force schedulers as they like, with different builders, properties, permissions, and so on. scopeTo make this project a summer's worth, it should aim to implement:
|
|
| #1788 | MQ options | |
| Description |
When Buildbot adopts an MQ framework, there are a number of choices. Among them:
The first is already implemented in 'nine', but we need more. The more difficult part of this project is to ensure that messages are only delivered after the corresponding database changes are visible. The issue is a race condition between message passing and database replication. Imagine you have a large Buildbot installation with several replicated MySQL servers and a redundant RabbitMQ cluster. One buildbot master writes changes to a build to the database (an UPDATE operation), then sends a message describing the change. On another master, some service gets the message and queries a different MySQL server to see the build's new status. If the message arrives before the database replication occurs, then this master will see stale data. That will lead to a lot of subtle, rare bugs. scopeThis project would involve separate tasks:
|
|
| #2218 | Add support for declarative configuration. | |
| Description |
Buildbot's configuration has historically been a Python file that Buildbot executes. The result of that execution is then used to set up the master. This assumes users can write valid Python code, and encourages people to do all manner of fragile or non-maintainable things. See #2218 for more. scopeThis project will involve some careful thought about how users interact with Buildbot, and how we can maintain backward compatibility while providing users with a forward migration path. Once that's complete, the modifications to the code will be fairly minimal, and mostly limited to master/buildbot/config.py (with associated tests). However, the change will require alteration -- a rewrite, really -- of the entire "Configuration" section of the manual; this will be the most time-consuming (but critical) part of this project. examples |
|
| #2435 | Complete Master-Side Source Steps | |
| Description |
GSoC 2011 included a project to add master-side source steps. These are a new implementation of steps that have historically housed their logic in the slave's code, putting that logic in the master instead. There were a few key pieces of functionality omitted from the new master-side source steps, and until that functionality is complete Buildbot is stuck maintaining *both* - an awkward position. scopeThis project would involve surveying the missing functionality, implementing it, and adding tests and documentation. This involves relatively little design work, but a good bit of hands-on debugging and coding. relevant bugsThe relevant bugs are tagged with 'master-side-source-steps:
|
|
| #2436 | Full Integration for User Support | |
| Description |
Buildbot deals with users in many ways - through version control, through the web, from IRC, from the command line for buildbot try and buildbot sendchange, and so on. In GSoC 2011, Derek Hurley added general support for correlating users across these interfaces, but the work remains unfinished. This project would involve adding the additional configuration and support for interacting with users:
scopeThis project has the advantage of starting from a solid base - there will be more coding than design here. It also breaks down nicely into individual tasks that can be completed one after the other, allowing you to scale the scope of the project up or down as the summer proceeds. Relevant Bugs
|
|
| #2437 | New Master-Slave Protocol | |
| Description |
Buildbot currently uses Twisted Python's perspective broker to communicate with slaves. This is a remote procedure call library that operates on TCP connections, and has a few disadvantages:
ScopeHere be dragons! Several people have attempted this before:
either of these may be a great starting point for this project, but at any rate this is a challenging project that will require a lot of thoughtful design work. The current plan is to use amp over ssh as the default slave protocol. It's probably worth looking at off-the-shelf Message Queuing projects like zeromq, RabbitMQ or Celery. The best approach is to find a way to get some working code put together quickly, while still allowing Buildbot to do everything its users expect. For example, if you can write your new protocol such that Buildbot can use perspective broker or your protocol, but you only get as far as a simple slave that can run echo hello world, that's great! We can ship that code, and someone else can pick up where you've left off to add more capabilities to your protocol -- assuming your design does not make that impossible. It would be great to select a communication protocol that is not Python-specific, so that a non-Python slave could be used to run Buildbot on more limited hardware (e.g., mobile devices). |
|
| #2438 | Windows Process Management | |
| Description |
The windows tag documents a number of bugs about starting and killing processes on Windows. Buildbot uses Twisted's process-handling code, so these bugs may be fixed by either re-implementing better support directly in Buildbot, or by patching Twisted's process-handling code. See also: ScopeTo do well with this project, you would need to bring a lot of the Windows experience that is lacking in the Buildbot development community. Assuming you're familiar with Windows APIs and accessing them from Python, this bug would entail
Related Bugs
|
|
| #2459 | full metrics support | |
| Description |
A while back, Chris AtLee? added some nice metrics code to Buildbot (#1945 / master/buildbot/process/metrics.py). However, the metrics it gathers are not accessible anywhere. The system gathers counts, times, running averages, and "alarms". scopeThis project would involve expanding that support, adding access to the metrics data via the Data API, and adding a flexible system to notify admins on alarms. It would also be helpful to alarms support a little more flexible, so it can be used to handle everything from logged tracebacks on a master to oversized builder queues to disconnected slaves. See the bugs below for related tasks. Relevant Bugs
|
|
| #2460 | Build a plugin architecture | |
| Description |
It should be possible for users to add extra functionality to Buildbot by installing additional packages from pypi. The extra functionality should include
The setuptools entry-points support can help here. Once this is complete, long-term unmaintained code currently distributed with Buildbot should be moved to external projects (still in the Buildbot github organization) where it can be maintained - or not - by its users. We should produce simple "hello-world" templates for each type of plugin, so that users have somewhere to start. scopeThis project will involve understanding how Python packages are built, tested, documented, and distributed. |
|
| #2461 | Add support for providing and graphing data charts of build statistics over time. | |
| Description |
This is a feature request I would find extremely useful, and it's also one that would probably be a good one for GSoC, since it's a new feature with visually concrete results, which can make it more compelling/tractable for a new developer to get interested in. I maintain a continuous testing architecture for the Emscripten C++->JS compiler project ( https://github.com/kripken/emscripten). It has an extensive unit testing and benchmarking suite, which prints a lot of graphable metrics out as the result of its run. The buildbot page is openly accessible at http://clb.demon.fi:8112/waterfall . More explanation at the bottom of the page here https://github.com/kripken/emscripten/wiki . While maintaining the buildbot, I've often wondered about the following:
To be able to measure these kind of quality concerns, visual graphing could be the answer. Being able to feed custom data fields into a data storage inside buildbot, and having a built-in data grapher integrated to buildbot HTTP server to visually compare metrics over time would be immensely helpful. The architecture should be somehow custom-driven so that the buildbot config files can control what data to generate and feed into graphs, since most of the above data points are specific to the project in question. Utilizing existing free graphing solutions would be expected, see for reference:
|
|
Your Idea Here!
We are open to any ideas that will make Buildbot better - think of this page as a set of ideas to get you started.