Buildbot master should have the ability to delete aging twisted logfiles

[bhearsum]

Buildbot logfiles can grow to be big fairly quickly. I think that the Buildbot master (and slaves?) should be capable of managing this. It would be nice to say "delete logfiles older than XXX" or "only keep logfiles for XXX number of builds" or "use a maximum of XXX GB for logs", etc.

[warner]

I was wondering about this too.. with the flat-file backend we have now, a simple external cron job is sufficient (mine just delete files in the builder directories that are more than a month old.. also note that you can delete the log and leave the main build file there, and then you still get basic pass/fail info).

But the upcoming SQL backend will render this approach useless, so the buildbot has to take an active role in limiting db size.

What sort of options should we have? I'm planning on having a c['database'] or cstatus_database?}}} object of some sort, so we can stick options on it.

[Pike]

Twisted can just do this itself,  http://twistedmatrix.com/projects/core/documentation/howto/application.html#auto6.

I don't think that the twistd.python.log has anything to do with status plugins, btw.

Taking this bug and moving it over to configuration.

I think I'm going to add a option to both create-master and create-slave to support log size and log count options.

That said, we should have a separate bug for build logs, i.e. step stdio buildbot log files. Gonna look for that one, too.

[Pike]

... sadly depends on Twisted 8.2.0,  http://twistedmatrix.com/trac/browser/tags/releases/twisted-8.2.0/NEWS?format=raw and  http://twistedmatrix.com/trac/ticket/3534

I'll go down that route nevertheless, sounds like the right way to do it.

[Pike]

git branch available on  http://github.com/Pike/buildbot/tree/bug108 for review.

I had to add a few markup fixes to the docs. I only added the options to the create-slave docs, as the create-master options are currently undocumented anyway.

There is a test for the master-side of this, which was already funky to do as it requires to actually start a real buildbot master and not just one of our fake ones. I didn't add a test for the slave, as that'd just be harder still, and it's mostly the same code.

[dustin]

Geez, 8.2.0 isn't much more than two weeks old yet!

As a minor quibble, I'd like for the new options to be specified as variables at the top of the .tac, just like basedir and configfile.

The only problem I have with this is that it hard-codes "twistd.log". I use

twistd .. --logfile=- ..

when debugging to see the log output in my terminal.

Is there a way to get access to the twistd command-line arguments in buildbot.tac? Alternately, is there a way to "adapt" the existing ILogObserver for an application, and extract the filename from it?

[Pike]

That logfile one is tricky. buildbot.tac is loaded from  http://twistedmatrix.com/trac/browser/tags/releases/twisted-8.2.0/twisted/application/app.py#L406 run method via self.application = self.createOrGetApplication().

After that, the logging is set up, and  http://twistedmatrix.com/trac/browser/tags/releases/twisted-8.2.0/twisted/application/app.py#L244 grabs the overwritten log.

The underlying issue is that buildbot.tac is just loaded in an eval within  http://twistedmatrix.com/trac/browser/tags/releases/twisted-8.2.0/twisted/persisted/sob.py#L192 loadValueFromFile, and the context for that just has basically nothing.

So I neither have the twisted commandline, nor is the logging set up yet at the time buildbot.tac is invoked.

I could hack that, and just hardcode searching --logfile= in sys.argv. That's kinda ugh-y, though, IMHO.

I'll do the variables thing in a follow up patch on the branch.

[dustin]

This seems like it would be a universal problem -- perhaps it makes more sense to fix it partway in buildbot, and put a "real" fix in twisted itself?

[dustin]

Replying to Pike:

I'll do the variables thing in a follow up patch on the branch.

I don't see this on github yet .. am I missing it?

[Pike]

Got taken away by some day-life patches in Mozilla land, I'll update here once I push.

[Pike]

Updated the 108 branch. Opened ticket on twisted on  http://twistedmatrix.com/trac/ticket/3615.

[dustin]

Great - merged to master.

[Pike]

I just saw that when running the tests without having a corresponding buildbot installed, the which('buildbot') of course fails. I don't have a good idea on how to fix that, so I caught that case and made it raise a Skip.

 http://github.com/Pike/buildbot/commit/a6b3d232b6aa9222bf877b8414c1affa909ae6f5 is on my bug108 branch.

[dustin]

Thanks - got it!