Opened 8 years ago

Last modified 4 years ago

#1795 new enhancement

add python script slave-side command

Reported by: dustin Owned by:
Priority: major Milestone: 0.9.+
Version: 0.8.3p1 Keywords: sprint
Cc: rutsky.vladimir@…

Description (last modified by Jc2k)

Complex slave-side functionality is often written as a big 'ol master-side shell script, e.g.,

https://github.com/buildbot/metabbotcfg/blob/master/builders.py

This works pretty well for systems with a shell (although you still need to deal with shell portability issues), but is problematic for Windows.

It's not particularly difficult to run Python scripts slave-side right now, either. Usually you'll use a FileDownload? to send the script, and then a ShellCommand? to invoke Python. That requires knowing with Python is installed, though, which varies from platform to platform. But the Buildbot-slave code is running in the python interpreter and knows exactly what its pathname is (sys.executable)!

So this is proposing adding a slave-side command that can take a string and execute it in a freshly-spawned Python interpreter on the slave.

Change History (8)

comment:1 Changed 7 years ago by dustin

  • Keywords sprint added

comment:2 Changed 6 years ago by tom.prince

  • Milestone changed from 0.8.+ to 0.8.8

comment:3 Changed 6 years ago by tom.prince

Hmm. This could be implemented simply by exposing sys.executable to the master, and then using ShellCommand to run it. With StringDownload to get the code there.

Or, we could pass the code to be executed with -c. One possible issue with this, is how newlines are handled. Some code on the twisted buildbot uses

from binascii import unhexlify; exec(unhexlify(b"%s"))

to avoid dealing with newlines et al in the code to run.

comment:4 Changed 6 years ago by dustin

Writing to a temp file (even with mkstemp) seems like a more durable solution. Command lines have limited length. And who knows what happens on Windows.

comment:5 Changed 5 years ago by krf

FYI: I had that working at some point by simply using base64 to encode the Python-code string on the master and decode it on the slave again. That pretty much worked on every platform I was targeting at that time (Windows, OSX, Linux).

comment:6 Changed 5 years ago by rutsky

  • Cc rutsky.vladimir@… added

comment:7 Changed 4 years ago by Jc2k

  • Description modified (diff)
  • Milestone changed from 0.8.9 to 0.8.+

comment:8 Changed 4 years ago by dustin

  • Milestone changed from 0.8.+ to 0.9.+

Ticket retargeted after milestone closed

Note: See TracTickets for help on using tickets.