Opened 2 years ago

Last modified 4 months ago

#3203 new enhancement

Support control messages via MQ

Reported by: dustin Owned by:
Priority: critical Milestone: 0.9.+
Version: master Keywords: mq

Description (last modified by dustin)


we need a way to communicate back and forth between the frontend web UI and backend schedulers (and other components).

This might be some kind of IPC over MQ.

Change History (8)

comment:1 Changed 2 years ago by dustin

  • Description modified (diff)

comment:2 Changed 2 years ago by dustin

  • Description modified (diff)

comment:3 Changed 22 months ago by dustin

I suspect some of this has been implemented -- @tardyp?

comment:4 Changed 22 months ago by tardyp

only stop is implemented via mq, because it is mandatory.

rebuild, triggereable, and force are implemented on the master that execute the control rest api (as of today)

Note that the goal of everything via mq is to allow for a www-only master to have a read-only creds to the db.

comment:5 Changed 22 months ago by dustin

For a 0.9.0 release, i think we need to have an understandable story about multi-master support. I'd really rather that not be "doesn't work" (since we have had decent multi-master support in 8).

Is there some simply way to describe what does and does not work right now?

comment:6 Changed 18 months ago by tardyp

A simple way to say it is by adding the concept of symetrics and asymetrics multimaster (like there is SMP and AMP for multi core CPUs)

Symetric multimaster is when each master share the exact same configuration. They run the same builders, same schedulers, same everything, the only difference is that slaves are connected evenly between the masters (by any means (e.g. DNS loadbalancing, etc)) Symetric multimaster is good to use to scale buildbot horizontally.

Asymetric multimaster is when each master have different configuration. Each master may have a specific responsability (e.g schedulers, set of builder, UI). This was more how you did in 0.8, also because of its own technical limitations. A nice feature of asymetric multimaster is that you can have the UI only handled by some masters.

Separating the UI from the controlling will greatly help in the performance of the UI, because badly written BuildSteps? can stall the reactor for several seconds.

asymetric multimaster is a more difficult goal to achieve, even if I understand the need, this should not block nine release

comment:7 Changed 18 months ago by dustin

  • Milestone changed from 0.9.0 to 0.9.+

OK, so symmetric multimaster (SMM) is working now, and AMM is a 0.9.+ feature.

Note: See TracTickets for help on using tickets.