refactor web status display

[dustin]

I'd like to write an alternative web-based status display, but waterfall kind of assumes it's the only thing in its URL space.

The attached patch refactors the web status quite a bit to:

• allow the various sub-pages (changes/*, $BUILDER/*) to be used outside of the waterfall
• allow multiple web status displays to run in the same Site at different sub-URLs

Outside of the Site-related changes (keeping html.Waterfall as a backward-compatibility function), there's not a lot of code change here.

[dustin]

factor-web.patch

[dustin]

This is preliminary -- no docs, no unit tests.

[warner]

Could you split this into two patches, one which refactor (but leaves the behavior the same), and a second which changes behavior? That would make it easier for me to figure out what's going on. My hunch is that this is a good direction to go, but I'm finding it a bit hard to be sure.

Oh, and on the how-to-test-it side, I came up with an approach for another project of mine ( http://allmydata.org/trac/tahoe/browser/src/allmydata/test/test_web.py): create a dedicated Status object in a temporary basedir, populate it with a couple of fake builds, attach it to a WebStatus? object, then do a bunch of client.getPage calls and examine their results.

[warner]

Actually, hold off on splitting this up, I think I'm getting a handle on what you're doing, plus I think I'm going to incorporate this functionality differently.

A couple of random notes on the code itself. The signature for WebStatus changed with the addition of top_page= : to maintain backwards compatbility, new arguments should be added at the end of the list rather than the beginning. I'd prefer to implement named_pages= with an add method rather than a kwargs that prevents us from ever adding new arguments in the future (and I'd name the method .putChild to match the same functionality in twisted.web.resource.Resource).

My plan is to get rid of setBuildmaster() and instead pass the buildbot.interfaces.IStatus (and IControl) through the request object. That will remove the need for doing setup at config time, and a lot of code goes away.

My other plan is to put a twisted.web.static.File at the root of the resource tree, with the buildbot-provided resources as static children of that root. The actual root resource (what you get if you hit, e.g.,  http://buildbot.buildbot.net/) will be a special thing that looks for an index.html on disk and serves it if present; if not it will redirect you to /waterfall . The idea is that for all of your robots.txt and .css files and other static files, you just drop them into BASEDIR/html/ , and then if you have multiple views you can write your own index and point to them from your welcome page.

Basically I want to combine this ticket with #17, and I think I'll use the presence of all this code as motivation to make some progress on #17. I'm not sure about whether this will go into 0.7.6 or something soon afterwards.. it would be awfully nice to have, but people have been waiting a long time for 0.7.6 too :).

[dustin]

I agree with your wish to make a release, vs. adding new features -- no problem there. This patch in itself doesn't add anything for users, so it doesn't make sense to get it into 0.7.6.

I'm happy to see this refactored any way you'd like -- I realize that my "hack" of delegating to top_page isn't the most elegant thing ever, and the idea of using a redirect sounds great. The aspects of this I need for my new display (which I'll upload shortly in another ticket, so you can take a look) are the following:

• ability to host mutiple status displays in the same URL space
• ability to place the various subpages (builders, changes, builds, logfiles, etc.) at sub-urls of any web status display.

So anything you invent that makes that happen will be great.

[warner]

I've been hacking on this one all day.. take a look at the web-parts branch (http://buildbot.net/repos/web-parts/) and tell me what you think.

The basic idea is that the WebStatus?() server provides the traditional Waterfall at a URL of /waterfall, and /builders is where all the current sub-pages live:

• /builders/$BUILDERNAME : lets you force builds, etc
• /builders/$BUILDERNAME/builds/$BUILDNUM : describes a specific build
• /builders/$BUILDERNAME/builds/$BUILDNUM/steps/$STEPNAME : describes a specific step
• /builders/$BUILDERNAME/builds/$BUILDNUM/steps/$STEPNAME/logs/$LOGNAME : shows a logfile
• /changes/$CHANGENUM : show a specific change
• /schedulers/$SCHEDULERNAME : view/manipulate a scheduler

These are all provided by WebStatus? code. Everything else is served from BASEDIR/public_html/ , so that's where /robots.txt, /favicon.ico, etc. all go. I'm still trying to work out some sort of simple templating system, so you can provide a header or footer on all generated pages.

This leaves room for other kinds of status displays. In particular, I'm working on some new ones (one-line-per-builder, RSS feed, embeddable IMG tag) that will live in this URL space. Also, you can use putChild() to add arbitrary resources into this URL space, so if you've created your own display, you can make it show up at /my-status-page with:

ws = WebStatus(8012)
ws.putChild("my-status-page", MyStatusPage())
c['status'].append(ws)

The main Waterfall page is acquiring some query-arguments to control what gets displayed. I'm working on adding branch=, which is how you'll restrict the display to only showing changes or builds that involve branches you care about. This way, Builders can go back to being about platform and process, and not also being about branch.

[warner]

This just got merged in to trunk, as of [4adfacbc56a2489ece50646459d59dd55ca9857b].