wiki:MasterSlaveCommunication

Version 5 (modified by tom.prince, 12 months ago) (diff)

Remove remote_ prefixes, as they are implementation details.

There is a new message-based protocol under development. This will replace the Twisted Perspective-Broker based RPC protocol.

Current Protocol

For the current API, See: Master-Slave API

The current protocol sequence diagram, if represented as messages, looks similar to this:

SequenceDiagram

Most remote calls are asynchronous.

Initial Interaction:

Authentication: Currently the slave authenticates with the Twisted Cred (newcred) mechanism, using the username and password supplied to buildslave create-slave.

This occurs as part of the setup of the  PB connection, and so doesn't fit into the RPC style of the rest of the current protocol. As it happens, the login mechanism is also used to allow the same port to be used for other PB communications.

After that the master requests information about the slave, via the remote methods getSlaveInfo, getVersion, and getCommands.

The SlaveInfo? currently consists of the contents of the slave's info/ directory.[[BR}} This also contains the keys
environ: copy of the slaves environment
system: OS the slave is running (extracted from pythons os.name)
basedir:base directory where slave is running

getVersion returns the version of slave. getCommands returns the list of (name, version) for all commands the slave recognizes. These methods do not take any arguments.

Next the master sets the builder list via setBuilderList. This method, given a list of builders and their build directories, ensures that those builders, and only those builders, are running. It returns a set of SlaveBuilder objects. This method can be called later as well.

Then the master sets itself as master for each SlaveBuilder via setMaster. This returns a reference to the master-side SlaveBuilder.

Reactor Loop:

The twisted event loop, or reactor, then waits for external triggers, such as Build.

On receiving the Build trigger, the master then runs commands on the SlaveBuilder. For documentation on commands see : Remote Commands
For each of the commands,
The command is started with startCommand. This takes stepref, stepId, command, args as arguments.
The command can be interrupted via an external trigger, which calls interruptCommand. This method takes stepID and a reason why as arguments.

The running command provides Updates, sent as arguments to update from the SlaveBuilder to the master, during execution.

At the end of a command, the completion of the command is indicated by calling complete, with failures as optional arguments.

The master and slave(in versions greater than 0.8.3) can initiate the shutdown of a slave. The master can call shutdown to shut down the slave, while the slave can also request for a graceful shutdown.

Which it does by calling shutdown on the master.

Proposed Protocol

The proposed protocol is message-based rather than RPC based. The protocol would be versioned. The initially planned changes are shown below. ProposedSequenceDiagram

Changes will include reducing the number of roundtrips, support for several masters, a busy-signal from the slave, protocol versioning and other improvements.

Attachments