Ticket #2260 (new defect)

Opened 14 months ago

Last modified 12 months ago

'builddir' path is used on both master and slave

Reported by: ayust Owned by:
Priority: minor Milestone: 0.9.0
Version: master Keywords:
Cc: dustin

Description

So as it turns out, a given builder's builddir property is actually used for two different things:

  • On the slave, it's the path to the directory which contains the builder data (sourcestamp, etc) and workdir
  • On the master, it's the path to the directory which contains the saved build results

For relative paths, this works "ok" most of the time - on the master, it's relative to the master's root directory (which is unique-per-master), and on the slave, it's relative to the slave's root directory (which is fine, because it's unique-per-slave, and slaves are currently unique-per-master).

Problems arise, however, if one of the following is true:

  • two different buildbot (masters/slaves) are running on the same machine, and an absolute path is used
  • two builders have the same builddir, even if they're on different slaves

Because the builddir path is also used on the master for storing results files, this introduces a potential conflict between paths for multiple builders (potentially even across different masters, in the case of absolute paths) which can result in race conditions and/or combination of data from different builds.

At a bare minimum, Builder should be changed to have separate path settings for the master-side data storage and the slave-side build directory.

Change History

comment:1 Changed 14 months ago by tom.prince

There is already both builddir and slavebuilddir, with the latter defaulting to the former.

comment:2 Changed 14 months ago by ayust

I'd say they should get renamed then, because I definitely expect builddir to refer to the slave directory first. resultsdir would make more sense for the master's version, and it's the one that should get set explicitly if different.

(workdir is relative to builddir, builders are primarily operating on slaves, etc.)

Last edited 14 months ago by ayust (previous) (diff)

comment:3 Changed 14 months ago by dustin

  • Keywords simple added
  • Priority changed from critical to minor
  • Milestone changed from 0.9.0 to 0.8.7

At one point, there were explicit checks for overlapping builddirs, though slavedirs are intentionally allowed to overlap (Use At Your Own Risk).

This check, assuming it's still working, would prevent the second point -- "two builders have the same builddir" -- from occurring.

Since this will be obviated by moving this data to the database, perhaps for the remainder of 0.8.x we should just issue a warning for absolute paths in builddirs?

  • minor because while this caused mayhem, it is something of a corner case; and
  • simple because adding this warning only requires grokking master/buildbot/config.py.

comment:4 Changed 12 months ago by Tom Prince

Add warning about absolute builddir.

Refs #2260.

Changeset: 516d41c436acf2e9fb08a7549c7beb499c2e70d6

comment:5 Changed 12 months ago by tom.prince

  • Keywords simple removed
  • Milestone changed from 0.8.7 to 0.9.0

We now warn about absolute paths, so moving this back to 0.9.0, when results will live in the db.

comment:6 Changed 12 months ago by Tom Prince

Add warning about absolute builddir.

Refs #2260.

Changeset: 516d41c436acf2e9fb08a7549c7beb499c2e70d6

Note: See TracTickets for help on using tickets.