Add load-balancing support

[ipv6guru]

When running lots of schedulers / builders on relatively few machines it becomes increasingly important to manage resources, i.e. strict the number of builders that can run simultaneously. A first step into that direction may be the use of semaphores to indicate how many builders may run in parallel on a given machine.

However, when managing multiple projects with multiple buildbot setups the above isn't enough either. Scheduling builds needs to take into account the 'outside' world, such as the current load of a machine. If there is only a single machine to which a builder is assigned, it may mean that a build has to be delayed. However, if there are multiple equivalent machines available, a single builder could be assigned to a class of equivalent machines, instead of a single machine, and the scheduler can then assign a build to the machine with the lowest load.

I indicated interest into such functionality in the past ( http://sourceforge.net/mailarchive/forum.php?thread_id=10215131&forum_id=3 2301) and even suggested to work on it (with some help), but didn't receive any feedback from other developers. That's pretty sad ! I'm not giving up just yet, though. :-) Submitted: Stefan Seefeld ( stefan ) - 2006-07-18 23:32

Additional Comment :-

Date: 2007-05-07 22:14 Sender: axel_hecht

Hide Not sure if we should move this over to trac, anyway:

I can't get to the initial post quoted here, thanks to sf.net :-/, but I have done one little tweak to process/builder.py to choose a random slave instead of the first in maybeStartBuild. That way, you at least distribute the load in the case of

1 Scheduler N Builders n Slaves

N being >> n.

Going more into load balancing was more involved than what's currently good for me, as that would require native code on the slave to get the load. And one had to decide whether disk or cpu load counts, etc. The fact that the SlaveBuilder? objects are per Builder didn't help either, that is, there's no information on how many Builders are trying to run per slave. Random thus seemed like a good first start.

[ipv6guru]

Moved from sourceforge

[stefan]

This is a duplicate of #42 (I should know, I submitted both ;-) ). I'd suggest to close this one, and to reset #42's milestone to 1.0.0 (from 1.0+), so it (finally) gets done for 1.0, not after it.

Thanks !

[dustin]

as directed..

[dustin]

Milestone 1.0.0 deleted