Opened 6 years ago

Last modified 6 years ago

#2359 new enhancement

Support extracting steps from annotated logs

Reported by: dank Owned by:
Priority: minor Milestone: 0.9.+
Version: master Keywords:
Cc: aki

Description

In general, putting too much knowledge about how builds are done into buildbot config files is considered harmful because it makes it hard for non-buildbot-admins to update or test the build procedure used by buildbot.

One approach to dealing with this is to have buildbot run a single step that just runs a script that is under full control of the developers (as opposed to buildbot admins). That's good, but then you lose visibility into build steps, and the waterfall is a bit more opaque.

Chromium currently does this, and solves the big-opaque-single-step-waterfall problem by parsing the logfile and synthesizing steps from annotations inserted into the log by the build script.

See http://lackingrhoticity.blogspot.com/2011/08/fixing-trouble-with-buildbot.html for discussion, and code and tests at http://src.chromium.org/svn/trunk/tools/build/scripts/master/chromium_step.py http://src.chromium.org/svn/trunk/tools/build/scripts/master/unittests/annotator_test.py

This seems like a Good Idea. Any chance it could be merged into buildbot proper?

Change History (8)

comment:1 Changed 6 years ago by tom.prince

I do like the idea of allowing the build config to be controlled by version controlled files. But I'm not sanguine about synthesizing steps, though. I think, mainly, because I think this would increase the api surface that could be considered public significantly.

Another approach to this design space is https://github.com/Jc2k/buildbot_travis

comment:2 Changed 6 years ago by dank

I don't understand how this would increase the public api surface. The calls that synthesize steps would be part of buildbot proper, and those functions could be marked 'internal, not for public use'.

comment:3 Changed 6 years ago by tom.prince

Well, so the issue is that right now, buildbot doesn't have a well-defined public api, and various configurations make assumptions about how buildbot works (which incidentally makes them hard to upgrade if those assumptions are invalidated, hence mozilla being stuck on 0.8.2 for example). The point is, that allowing steps to be synthesized means that steps can exist with state that have never been run, and steps are created in the middle of a build, and probably other things that I haven't thought of. And so code that deals with those things needs to be prepared to handle those situations. That is what I meant by API surface, which still exists, even if it is not reflected in a particular public method.

comment:4 Changed 6 years ago by dank

The key advantage of the annotations technique is that it's totally tool-agnostic. It doesn't require developers to change how they do local builds. It only requires them to add echo or print statements or whatever their tool uses to jam text into the log file.

The travis approach requires them to change how they build, and is likely to be rejected by the average developer.

comment:5 Changed 6 years ago by tom.prince

I'm not against something like this in general, nor am I saying that buildbot_travis is the proper solution. I am just leary of the implementation pointed at here, and am pointing out other approaches in the same design space.

Also, for inclusion into buildbot, I am wary of possible ambiguity in the parsing. For examplke if there was attempt to use it for metabuildbot (where the the control text may also occur in the output).

comment:6 Changed 6 years ago by dustin

I agree with tomprince. I think this is a good idea (it's been discussed before, but I didn't know it was implemented!), but will be better to merge in the 0.9.x series (when at least the status-reporting API is well-defined) or even 1.0.x (when process APIs get nailed down).

Throughout 0.8.x, we've adopted a lot of interesting tools that turn out to create edge cases, bugs, or contradictions after they're merged. I'd rather *avoid* doing so again in the 0.8.x series, so that we can focus on defining things properly in 0.9.x. Once those definitions are well understood, we will have the tools to properly analyze bigger changes like this, and identify any violated assumptions or unintended consequences.

We should think of this as a design goal as we build those releases, though.

comment:7 Changed 6 years ago by dustin

  • Milestone changed from undecided to 0.9.+

comment:8 Changed 6 years ago by dustin

  • Cc aki added

This would interface very well with Aki Sasaki's Mozharness.

Note: See TracTickets for help on using tickets.