Opened 7 years ago

Last modified 4 months ago

#301 assigned enhancement

Allow a single buildslave to service multiple buildmasters

Reported by: dustin Owned by: sridhar
Priority: major Milestone: 2.0.+
Version: 0.7.7 Keywords:
Cc: dwlocks, Pike, kovarththanan.rajaratnam@…, ewongbb@…

Description (last modified by dustin)

We run a few different buildmasters -- they're monitoring different repos, and in general are for different teams. But we use the same (large) set of build VMs for both. Currently we run two buildslave instances on each build VM, which is inefficient in a few ways: it's a lot of duplicated data in RAM, since Python bytecode isn't shared the way .so's are; and it means that builds for two different projects can be taking place simultaneously on the same VM.

It would be nice to have the option of pointing a single buildslave at multiple buildmasters, with some sort of locking to only allow it to run a single build at a time.

Change History (18)

comment:1 Changed 7 years ago by bhearsum

Out of curiosity, is there a reason you cannot just combine the two masters?

comment:2 Changed 7 years ago by dustin

Besides being for different projects, they have completely different ChangeSources?, schedulers, etc.

That might be an easier modification to make, though -- let two buildmaster instances run in the same Python process..

comment:3 follow-up: Changed 7 years ago by Pike

  • Cc Pike added

I'd love to see a way to partition a buildmaster, too. I'd still like to be able to punch little holes between the partitions, just having a sub-bunch of changesources go to a sub-bunch of schedulers would probably suffice for me.

Dustin, would you guys need to have independent master.cfg?

comment:4 in reply to: ↑ 3 Changed 7 years ago by dustin

Replying to Pike:

Dustin, would you guys need to have independent master.cfg?

Not necessarily -- Python's "import" mechanism could achieve whatever sort of separation we would need.

I'm thinking about how to do this. We'll see what I come up with.

comment:5 Changed 6 years ago by dustin

  • Milestone changed from undecided to 1.0.0

comment:6 Changed 6 years ago by sridhar

Just throwing ideas, can have something like this:

$ buildbot create-slave BASEDIR MASTERHOST1:PORT1,MASTERHOST2:PORT2,... SLAVENAME PASSWORD

buildbot would create directories MASTERHOST1_PORT1, MASTERHOST2_PORT2,... under BASEDIR. From then on, it would work exactly like it works now. Buildbot client will just be talking to multiple masters and jump into various MASTERHOST_PORT directories to do the usual work.

The first iteration will have the slaves servicing one master at a time, but can later have them servicing multiple masters in parallel.

comment:7 Changed 6 years ago by dustin

  • Milestone changed from 1.0.0 to 0.7.+

This sounds like a great idea to me. Go for it!

comment:8 follow-ups: Changed 6 years ago by Pike

I'd like to see slave locks and max_builds work in this scenario, fwiw.

comment:9 in reply to: ↑ 8 Changed 6 years ago by sridhar

Replying to Pike:

I'd like to see slave locks and max_builds work in this scenario, fwiw.

I have no idea about those, but will look into it and try to get them working. Is there any deadline/ETA?

comment:10 Changed 6 years ago by dustin

0.7.11 is scheduled for May 25.

comment:11 Changed 6 years ago by sridhar

  • Owner set to sridhar
  • Status changed from new to assigned

comment:12 in reply to: ↑ 8 Changed 6 years ago by sridhar

Replying to Pike:

I'd like to see slave locks and max_builds work in this scenario, fwiw.

I assume that the masters need not be aware of the other master's setting they are sharing their slaves with. If masters X and Y are sharing a slave and have max_builds=x and y respectively, should: 1)max_builds=min(x,y) and the current number of build is the number of builds issued by X and Y. 2)the earlier definition of max_builds still holds and both the builds are run from different directories with their own set of parameters. This would mean that each master would see that it has the slave to itself with its own max_builds and locks .. however this would mean that the person setting up the masters would also need to be aware of the fact when setting masters up.

I prefer the first approach.. however I might be overlooking some of the use cases. Thoughts?

comment:13 Changed 5 years ago by dustin

  • Milestone changed from 0.8.+ to 1.0.+

This is still a good idea, but it is quite hard. It's probably best implemented along with remsh.

comment:14 Changed 4 years ago by krajaratnam

  • Cc kovarththanan.rajaratnam@… added

comment:15 in reply to: ↑ description Changed 4 years ago by tom.prince

A really stupid way to do this is right now is just edit the .tac file to start multiple slaves. Obviously, they don't communicate at all, and they probably handle signals, and shutdown requests strangely. Starting multiple bots in BuildSlave would be rather trivial. The tricky part is figuring out how to handle locks and shutdown requests.

buildmaster_host = 'master1'
port = ...
slavename = 'slave_name'
passwd = '...'
keepalive = 600
usepty = 0
umask = None
maxdelay = 300
slavedir = 'master1'

s = BuildSlave(buildmaster_host, port, slavename, passwd, os.path.join(basedir,slavedir),
               keepalive, usepty, umask=umask, maxdelay=maxdelay)
s.setServiceParent(application)

buildmaster_host = 'master2'
port = ...
slavename = 'slave_name'
passwd = '...'
keepalive = 600
usepty = 0
umask = None
maxdelay = 300
slavedir = 'master2'

s = BuildSlave(buildmaster_host, port, slavename, passwd, os.path.join(basedir, slavedir),
               keepalive, usepty, umask=umask, maxdelay=maxdelay)
s.setServiceParent(application)

comment:16 Changed 20 months ago by asibilev

Was this problem resolved somehow in any release? I mean out-of-box solution. Does anybody have experience this using single slave for 2 masters with locks?

comment:17 Changed 20 months ago by dustin

  • Description modified (diff)

There's been no action, that I know of, on this bug.

comment:18 Changed 4 months ago by ewong

  • Cc ewongbb@… added
Note: See TracTickets for help on using tickets.