Web Status: Grid

[dustin]

Add a new web display a grid of build statuses organized by SourceStamp? on the horizontal axis and builder on the vertical axis. The display can be conditioned on branch and/or builder category.

To my mind, this is a much more useful view for developers -- rather than "what happened when" (the waterfall), it shows which revisions failed or were successful on which builders.

This patch is fairly well-developed, but I'm still making a few tweaks, so I may send an updated version or two shortly.

[dustin]

Screen Shot

[warner]

sweet!

[warner]

I like this. When you get a chance, update to HEAD and see if your patch still applies cleanly. If so, I'll merge it in. I'd like to see this attached unconditionally to the WebStatus? tree at a URL of, say, /status-grid . It should accept query-args of builder= and branch= to limit the display to specific builders and branches (although don't sweat the branch= arg, I haven't finished adding that to the waterfall yet, so I'm not sure how much work it is.. if you don't add branch=, I'll go ahead and add it later).

[dustin]

It already has branch= and category=, but no builder=. I can add that pretty easily.

Actually attaching this to a branch is going to take me sitting down for a while to learn darcs -- maybe this weekend. I can probably get an updated patch (against CVS HEAD) to you today, though.

[dustin]

Replying to dustin:

Actually attaching this to a branch is going to take me sitting down for a while to learn darcs -- maybe this weekend. I can probably get an updated patch (against CVS HEAD) to you today, though.

Well, I learned darcs and set it up locally, but didn't get a chance to actually incorporate the grid status.

It seems that the web-parts branch is fairly non-configurable at this point. Rather than add all of these neat new status displays at fixed URLs, why not make the existing behavior (waterfall at the root) the default for those with existing configs, and devise some easy way for those who are interested to add these status displays, parameterized on builder, category, branch, etc.

I'll see if I can get some more time to work on this during the week.

[warner]

Mostly because I've been in a very RESTful frame of mind the last few months, and I like the idea of having all these status displays be available all the time. The buildmaster admin might not think to enable, say, the statusgrid page, but that doesn't need to prevent users from getting to it.

The admin is free to have the index.html file provide a redirect to one of the other displays, or to set up a splash page that invites users to follow some specific links. They can also use putChild() to drop specific resources at specific top-level URLs, but my general thought is that less variability (== more consistency) is better. I'm willing to be convinced otherwise, though. When I first started working on web-parts, I was certainly thinking of a palette of resources and a master.cfg mechanism to attach those resources to different URLs. At this point I'm leaning more in the direction of having all resources be available all the time, and using index.html to provide links to the usefully-parameterized URLs.

BTW, the web-parts branch is now obsolete: it has been merged into trunk. I should remove that branch from the set of published darcs repositories tonight.

The now-legacy Waterfall puts a copy of /waterfall at the URL root, because that was easiest. I'd prefer to have it do a redirect, but I had problems making it do redirects to a relative URL.

[dustin]

Sounds fine to me -- will you be merging the grid, then, or me? Soonest I'll be able to get to it is Wednesday evening..

[warner]

I might get to it tonight, but if I haven't mentioned anything in this ticket, assume that I wasn't able to, and go ahead and hack away.

[warner]

I haven't been able to put any time into this, but I just finished adding a couple of helper functions that you might want to use. IStatus.generateFinishedBuilds is a generator that will produce IBuildStatus objects, starting with the latest, and working backwards in time. Your status grid class could keep pulling builds from it (and correlating them by revision) until it decided it had seen enough. IBuilderStatus.generateFinishedBuilds is similar but only produces builds from a single Builder: you could use a bunch of them in parallel and stop when you'd seen enough.

Both methods accept num_builds= to put a cap on the number of builds that will be returned. Both methods also accept branches=, which you should add as a query argument (see one_line_per_build for an example) to let people see a grid that only reflects trunk, or only reflects a specific branch. The IStatus method also accepts builders=, which lets you only pay attention to specific builders (and again the statusgrid should accept query arguments to control what the rows are).

Hm, and while I'm thinking about it, there should also be query arguments to control how many revisions get shown, and another one to transpose the grid, so that people who have a lot of builders (but only care about a few revisions) can use their screen real estate just as well as folks who have only a few builders (but care about a lot of revisions).

The direction I'm going with web-status is for all of the status pages to be configurable with query-args, then let admins construct hyperlinks in their site's index.html to the pages that they think are most useful. I'm planning to add "About This Page" links to each status page to explain what sort of query-args could be added, so that end-users can construct their own links.

[warner]

I was hoping to get this into 0.7.6, but it's just slightly too much work to get in before the release (which ought to happen tomorrow). I've gotten it about 60% rewritten for the new framework, so with luck it I can get it done early in the 0.7.7 cycle.

(with the new framework, about 70% of the code goes away, since it doesn't need to be its own standalone webserver)

[warner]

incomplete patch using the new webparts framework

[warner]

no progress on this yet, bumping to 0.7.8

[dustin]

This requires the patch in #71, so that should be committed first.

I'm in the process of upgrading our install to your HEAD, so I'll send a patch once I've got that worked out. Any reason this can't fit into 0.7.7?

[warner]

oh, if we make enough progress on it soon, it can go in.. I'm just trying to triage tickets in the hopes of making an 0.7.7 sooner rather than later. I've got some time this week (on vacation), so I'm hoping to get a release out before I go back into work.

[dustin]

updated patch against HEAD; grid.py will be attached separately

[dustin]

This is an updated statusgrid display that works in my testing. I changed a few other things, too:

• added a footer on all pages (for the [welcome] link, if nothing else)
• changed everywhere that redirected to the waterfall to go somewhere else (I don't want my users to see the waterfall at all)

This still requires #71 be committed first.

[dustin]

I somehow missed a "from buildbot import util" in buildbot/status/web/build.py. Please add manually.

[dustin]

grid.py, slightly updated to process branches correctly

[dustin]

#72:statusgrid.patch in  http://darcs.r.igoro.us/buildbot/dustin/

[dustin]

Ivan Chelubeev pointed out that I was still using "specific" instead of "absolute" sourcestamps. He provided a patch to fix it (#72:grid-absolute.patch).

[dustin]

this is in brian's tree