__group__,ticket,summary,component,milestone,type,created,_description_,_changetime,_reporter ,1871,Goofy quoting/space handling in buildbot_service.py,,0.8.+,defect,2011-03-12T07:12:31-08:00,"If the slave directory has spaces in it, BB gets completely confused if following http://trac.buildbot.net/wiki/RunningBuildbotOnWindows It appears that the directory gets broken up on space boundaries, and if you explicitly add double-quotes to prevent that, the entire string (including quotes) is interpreted as a path _relative_ to buildbot's installation in site-packages/ That wiki page should include 2 prominent warnings about this pitfall until the actual problem is fixed.",2011-07-21T20:53:36-07:00,dabrahams ,1773,deprecate 'default branch',,0.8.+,enhancement,2011-01-19T14:46:28-08:00,"The concept of a 'default branch' causes huge pain, with no gain. Let's see if we can make it go away: * ChangeSources should always set {{{c.branch}}} to something * schedulers should require an explicit branch or branches * Source steps should require an explicit branch or branches For backward compatibility, all of these should have parameters that can make them act the old way.",2012-12-09T16:27:19-08:00,dustin ,149,addLogObserver and progressMetrics,configuration,2.0.+,enhancement,2007-12-09T16:57:24-08:00,"Brian Warner wrote: > On Sat, 24 Nov 2007 13:06:32 -0500 > ""Sergey A. Lipnevich"" wrote: > >> In 0.7.6, adding log observer that produces new metrics is tricky >> because each new metric must be also mentioned in build step's >> progressMetrics member. Would it be better instead to declare all >> metrics provided by log observer in its class, and automatically add >> new metrics names to progressMetrics once the observer instance is >> added? I can file a ticket and submit a patch, I'm asking here first >> if it's a good idea to make this change. > > I think that's a good idea. You'll need to double-check that the > ""expectations"" are still filled in correctly.. it's been a long time since I > looked at that code, but I seem to remember there is a place where we run > through all steps and fill them with expectations from the previous build. > We'd need to make sure that we enumerate the metrics before doing this > setting-expectations phase.",2013-01-26T12:37:07-08:00,sergey ,674,Give better caching directives for static files in web interface,,0.9.+,enhancement,2010-01-06T15:48:18-08:00,"It looks like my web browser is requesting `/default.css` and `/bg_gradient.jpg` every single time I switch to a new page. Even if I use back and forward buttons, which don't cause the ''page'' to be requested again, it still requests again those two files. The requests properly use `If-Modified-Since`, and buildbot properly replies with `304 Not Modified`, but it's still annoying. Clutters the webserver log, for one. According to [http://redbot.org/ redbot], the responses are “stale” as soon as they go out. They should have an appropriate `Expires` header, or `Cache-Control: max-age=3600` (1 hour expiration from now).",2010-12-06T14:16:24-08:00,Nicolas Callek,438,Mercurial does not update to the right revision when building a tag from a named branch,other,0.8.+,defect,2009-02-27T03:42:05-08:00,"This morning I ran into a very big issue with the way buildbot do a Mercurial checkout. Here is the context : * The repository contains a named branch * A tag has been set on a revision of that branch * From the buildbot web ui, we force a build with the tag as the revision * buildbot do a ""hg clone --rev THETAG "" * The actual current changeset of the clone after checking out is NOT the revision corresponding to the tag. Instead it update to the latest revision that is on the ""default"" branch. In the end the build is not done on the correct revision. I suggest not to rely on the ""clone --rev"" but rather do a ""pull"" followed by a ""update -r"". Additional reasons for that : * It will be way faster for big repositories (ours is ~130M). * When using clone --rev, the tag is ""lost"" because the .hgtags file is set in a later changeset than the tagged changeset. Note that I have reproduced the issue with buildbot 0.7.9 and 0.7.10, and Mercurial 1.0.1 and 1.1.2",2012-09-22T21:41:07-07:00,cdevienne Callek,2176,"buildslave hangs trying to kill process after ""1200 seconds without output""",,0.8.+,defect,2012-01-16T05:02:26-08:00,"The buildbot logs show the usual message: `command timed out: 1200 seconds without output, attempting to kill` looking at the console window of the machine that's running the buildslave.bat, we see a message: `ERROR: The process ""None"" not found.` Do you know where this message is coming from? Could it be that buildbot is trying to kill a process that's already died? It seems that the ""attempting to kill"" message is the last one that makes it to the logs - Looking through the code in `runprocess.py`, that doesn't make any sense - it seems to me that there's no way of getting through that function without hitting at least one other `log.msg` call... weird. anyway, this hangs the build, and we're forced to go in and reboot the buildslave machine. that then produces one final line in the logs: `remoteFailed: [Failure instance: Traceback (failure with no frames): : Connection to the other side was lost in a non-clean fashion.]` No doubt we should try and write better test code that doesn't cause the 1200 second timeout, but still, it would be good if buildbot didn't hang... additional info: * buildbot-master is running on debian * buildslave is running windows vista * seems to be an intermittent problem - maybe one in 5 runs? * we're using buildslave to run selenium webdriver tests, driven from python 2.7 ",2012-09-22T22:32:49-07:00,hjwp ShriramK,2463,SVN export does not recieve username/password,,0.8.8,defect,2013-03-05T06:38:25-08:00,"the 'svn export' command isn't being run with _dovccmd, so it's not getting the username attached. This most readily manifests itself when running the svn export step in a build started by the anybranch scheduler, triggerable scheduler or by rebuilding.",2013-03-05T07:07:38-08:00,cscott ashcrow,356,doxygen build step,other,0.8.+,enhancement,2008-09-24T07:27:00-07:00,At one point I was using doxygen for documentation. While I'm not longer using it I figured the build step could be useful for others. Attached is the buildstep.,2013-02-28T16:48:50-08:00,ashcrow bdbaddog,2321,P4Source: support client syntax,,0.8.+,enhancement,2012-06-19T08:41:05-07:00,"[http://www.perforce.com/perforce/doc.current/manuals/cmdref/o.fspecs.html Perforce path] can be specified in ""client syntax"". Currently P4Source's p4base argument seems to expect ""depot syntax"". Right now, p4base argument is used for two different things: as an argument to ""p4 changes"", and to filter output of ""p4 describe"". The first can be in any syntax, but the second must be in depot syntax since that's what Perforce outputs. So if one specifies p4base in client syntax, no output matches and no change is added. I think two should be specified by two separate arguments, probably with a sensible default.",2013-01-26T16:28:46-08:00,sanxiyn dustin,2378,when is a deprecated argument to master.addChange but that is how web hooks pass in changes,,0.9.0,defect,2012-10-02T10:26:27-07:00,,2013-03-02T09:15:44-08:00,tom.prince dustin,1890,debug Buildbot's memory use,,0.9.+,enhancement,2011-03-17T12:23:11-07:00,"I learned about a few tools at PyCon that may be useful for this purpose. First, we should document these tools in the developer documentation so that others can work with them, too. Second, we should apply them in development to see if there are any low-hanging fruit in terms of memory consumption. * sys.getsizeof (2.6 only) * Meliae - dumps a reference graph in JSON * gdb-heap - dumps the Python heap in GDB (mostly totals)",2013-02-26T19:21:02-08:00,dustin dustin,1993,"gather metrics on caches, move buildbot.cache to buildbot.process.cache",,0.9.+,enhancement,2011-06-18T20:49:12-07:00,,2013-02-26T19:27:44-08:00,dustin dustin,2479,master creation fails with nine if buildbot-www is not installed with pip,,0.9.0,defect,2013-03-21T13:46:12-07:00,"When I was creating a development environment for buildbot nine, I installed buildbot-www using 'python setup.py install'. However, if I tried to create a master using that setup I hit the following traceback: {{{ (sandbox)[tflink@localhost buildbot_work]$ buildbot create-master master mkdir /home/tflink/buildbot_work/master creating /home/tflink/buildbot_work/master/master.cfg.sample populating public_html/ populating templates/ Traceback (most recent call last): File ""/home/tflink/buildbot_work/sandbox/lib/python2.7/site-packages/twisted/internet/defer.py"", line 1214, in unwindGenerator return _inlineCallbacks(None, gen, Deferred()) File ""/home/tflink/buildbot_work/sandbox/lib/python2.7/site-packages/twisted/internet/defer.py"", line 1071, in _inlineCallbacks result = g.send(result) File ""/home/tflink/buildbot_work/src/master/buildbot/scripts/create_master.py"", line 145, in createMaster yield createDB(config) File ""/home/tflink/buildbot_work/sandbox/lib/python2.7/site-packages/twisted/internet/defer.py"", line 1214, in unwindGenerator return _inlineCallbacks(None, gen, Deferred()) --- --- File ""/home/tflink/buildbot_work/sandbox/lib/python2.7/site-packages/twisted/internet/defer.py"", line 1071, in _inlineCallbacks result = g.send(result) File ""/home/tflink/buildbot_work/src/master/buildbot/scripts/create_master.py"", line 128, in createDB master = BuildMaster(config['basedir']) File ""/home/tflink/buildbot_work/src/master/buildbot/master.py"", line 82, in __init__ self.create_child_services() File ""/home/tflink/buildbot_work/src/master/buildbot/master.py"", line 146, in create_child_services self.www = wwwservice.WWWService(self) File ""/home/tflink/buildbot_work/src/master/buildbot/www/service.py"", line 36, in __init__ for ep in pkg_resources.iter_entry_points('buildbot.www') ] File ""/home/tflink/buildbot_work/sandbox/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py"", line 1954, in load File ""/home/tflink/buildbot_work/sandbox/lib/python2.7/site-packages/buildbot_www-0.8.7p1_778_g98e77e1-py2.7.egg/buildbot_www.py"", line 59, in ep = Application() File ""/home/tflink/buildbot_work/sandbox/lib/python2.7/site-packages/buildbot_www-0.8.7p1_778_g98e77e1-py2.7.egg/buildbot_www.py"", line 52, in __init__ self.version = open(verfile).read().strip() exceptions.IOError: [Errno 2] No such file or directory: '/home/tflink/buildbot_work/sandbox/share/buildbot/dist/buildbot-version.txt' }}} Once I installed buildbot-www with pip, I was able to create a master without issue",2013-03-21T17:44:06-07:00,tflink dustin,1729,"Need indices on sourcestamps.branch, sourcestamps.revision",,0.9.0,enhancement,2010-12-10T11:27:11-08:00,"Without more indices on these tables, writing queries against this table is very painful. Maybe want to index project and repository while we're at it.",2013-03-02T09:18:32-08:00,catlee jaredgrubb,928,Allow builder to be associated with multiple categories,,0.9.+,enhancement,2010-07-22T07:40:17-07:00,"I'd like to be able to associate builders to multiple categories, so they can appear in multiple views in the web status UI. (I'e like to classify along multiple axes, such as ""internal"" vs. ""public"", as well as ""stable"" vs. ""devel"".",2013-02-26T18:33:36-08:00,stefan jaredgrubb,1688,there should be some way to retry builds that fail in setupBuild without submitting new Changes.,,0.9.0,enhancement,2010-12-01T12:30:47-08:00,"We hit an issue on a master recently where a reconfig failed part way through and caused a bunch of our custom code to have bad references in it. In turn, this caused a great number of builds to fail with a setupBuild exception. Even when a setupBuild exception occurs for another reason it really sucks that the builds never start and are never retried. I think that Buildbot should either retry them automatically are make it a lot easier to retry them. Because the builds never started we wouldn't be able to make use of the RETRY status to make this happen. I was chatting with Catlee about this and he suggested that we might be able to accomplish this by running newBuild() in a try/except block and resetting claimed_at/claimed_by if it fails, possibly after waiting some period of time.",2013-03-02T07:53:15-08:00,bhearsum sridhar,301,Allow a single buildslave to service multiple buildmasters,other,2.0.+,enhancement,2008-06-15T15:04:35-07:00,"We run a few different buildmasters -- they're monitoring different repos, and in general are for different teams. But we use the same (large) set of build VMs for both. Currently we run two buildslave instances on each build VM, which is inefficient in a few ways: it's a lot of duplicated data in RAM, since Python bytecode isn't shared the way .so's are; and it means that builds for two different projects can be taking place simultaneously on the same VM. It would be nice to have the option of pointing a single buildslave at multiple buildmasters, with some sort of locking to only allow it to run a single build at a time.",2013-04-25T05:29:44-07:00,dustin toinbis,174,Persist ETA data across restarts,other,0.9.+,enhancement,2008-01-28T08:58:28-08:00,"Currently Buildbot forgets the historical data it needs to compute ETA whenever it's restarted or (I think) reconfigured. According to Brian Warner in a buildbot-dev posting, this should not be too hard to fix: > The current scheme would require storing a dict of (metricname -> > expectation) for each step. The builder pickle would be the most appropriate > place for this. ",2013-03-02T08:51:38-08:00,gward toinbis,738,Record true start/end time for build steps,,0.9.+,enhancement,2010-03-09T16:39:30-08:00,"Buildbot currently records the start/end time for steps on the master. This means that end time of step N is always the same as the start time of step N+1. When master load is high, the latency between a slave actually finishing a command and receiving the next one can be significant. One idea is to have the slaves record the command start time internally, and then when the command is done, send the slave start/end time to the master. The master would use its version of the start time as the step start time, and then calculate the delta between slave start time as master start time as clock scew/network latency and add it to the slave's end time.",2013-03-02T09:06:08-08:00,catlee toinbis,796,Implement UI elements: build's progress bar & build time charts,,1.0.+,enhancement,2010-04-13T03:26:51-07:00,"Couple of UI elements could be implemented in order to make buildbot's webstatus pages be more intuitively-informative: a) build's progress bar. A graphical representation for the build's ETA time. b) build's timing chart, probably with distinct type(colour) of lines representing each step. jQuery(UI) + Plot(http://code.google.com/p/flot/) are the tools to be used initially, whilst having eyes wide open for the more appropriate ones (more possible options are described here: http://www.reynoldsftw.com/2009/02/6-jquery-chart-plugins-reviewed/). These improvements shouldn't affect the way JS-less browsers renders web status pages. External loading of JS libraries and/or caching should be considered to reduce an extra load of buildmaster. But that's already the scope of another ticket. ",2013-02-28T17:09:35-08:00,toinbis tom.prince,68,New mechanism for monitoring buildbot startup,configuration,0.8.8,defect,2007-07-30T17:18:29-07:00,"The buildbot slave default timeout can be too short for us. Sometimes, 5sec isnt always enough, and we get following: % cltbld$ buildbot start /Users/cltbld/macosx-slave1 Following twistd.log until startup finished.. The buildmaster took more than 5 seconds to start, so we were unable to confirm that it started correctly. Please 'tail twistd.log' and look for a line that says 'configuration update complete' to verify correct startup. % ... even though slave started ""normally"", responds fine to pings from buildbot master, and does handle jobs just fine. ",2013-03-02T07:46:42-08:00,joduinn tom.prince,921,buildmaster logging should be more segregated,,0.8.+,enhancement,2010-07-19T05:57:39-07:00,"In situations where many builders are running at the same time the master side log gets very confusing when trying to trace the execution of a single build. I think it would be much more sensible to segregate different types of logs on the master. It should be configurable, of course, but for a default I'd SWAG: * Each builder (possibly) build gets its own log file * Everything else goes in twistd.log (or whatever the name ends up being). I think it would be nice to allow ChangeSources and BuildSlave events to have their own log files too, but I don't think they are busy enough in the general case to make that the default behaviour.",2013-02-26T18:35:12-08:00,bhearsum tom.prince,2228,Allow specific ForceSchedulers to be associated with specific WebStatuses.,,0.9.0,enhancement,2012-03-04T20:58:20-08:00,"If different webstatus instances are available to different people, they should get different force build options. ",2013-02-28T19:26:38-08:00,tom.prince warner,540,overhaul ETA calculation,other,0.9.+,enhancement,2009-03-31T01:35:01-07:00,"The Progress and Expectation classes are overdue for a rewrite. They need to move to a scheme that I'll call Reverse Temporal Mapping because it sounds clever. The idea is that each StepProgress metric (an axis along which progress can be measured, like bytes of output or test cases run) should be sampled periodically during the Step. Nominally we'd like about 100 samples total, but if we have no idea how long the step will run then we can just sample every 10 seconds or something. Those samples form a graph of progress-versus-time. We then invert the graph to give us one of time-versus-progress. This is used to create an interpolation table that will map from progress values into elapsed seconds. During the next build, we'll take the current progress value and use this table to estimate how many seconds are left to go. The tables should be averaged over multiple builds, to provide some smoothing. The ETA values from the various metrics should be combined with some sort of weighted average (some metrics may be more accurate than others). We also need to handle overdue builds better. It may be useful to report both the ETA and the percentage complete (since a slow build that still produces the same output as before will show <%100 complete even when the build is overdue). It would also be useful to add some intstrumentation that lets us check the accurary of the ETA code, perhaps a table to show how the ETA changes as the build runs, comparing it to the actual elapsed time. The ETA should get more accurate as the build nears completion: it would be nice to see a graph of this convergence. Submitted: Brian Warner ( warner ) - 2005-05-18 05:19",2013-03-02T08:51:47-08:00,ipv6guru warner,24,save build status in an SQL database,statusplugins,0.9.0,enhancement,2007-02-17T16:24:53-08:00,"Some things would be made easier if we were to store build status in a database that could be read by other processes. SQLite would be a good starting point. ",2013-01-22T11:15:27-08:00,warner