Ticket #2212 (closed defect: fixed)

Opened 15 months ago

Last modified 15 months ago

Property placeholder does not work in shell command

Reported by: DavidIAm Owned by: tom.prince
Priority: minor Milestone: 0.8.+
Version: master Keywords: properties
Cc:

Description

This configuration fragment:

tfactory = BuildFactory()
    tfactory.addStep(SVN(
                       svnurl=url,
                       mode='update'
                       ))
    tfactory.addStep(ShellCommand(env={'BUILD_NUMBER': Property("buildnumber")}, command=["make",target]));

Change History

comment:1 Changed 15 months ago by DavidIAm

It appears that it is only allowing placeholders such as as a string containing ${buildnumber} through a regular expression (buildbot_slave-0.8.5-py2.7.egg/buildslave/runprocess.py:297)rather than allowing possible number value to be used as an environment value

comment:2 Changed 15 months ago by DavidIAm

Resulted in this traceback:

remoteFailed: [Failure instance: Traceback from remote host -- Traceback (most recent call last):
  File "/home/dihnen/tmp/buildbot/sandbox/local/lib/python2.7/site-packages/Twisted-11.1.0-py2.7-linux-x86_64.egg/twisted/spread/pb.py", line 910, in _recvMessage
    netResult = object.remoteMessageReceived(self, message, netArgs, netKw)
  File "/home/dihnen/tmp/buildbot/sandbox/local/lib/python2.7/site-packages/Twisted-11.1.0-py2.7-linux-x86_64.egg/twisted/spread/flavors.py", line 114, in remoteMessageReceived
    state = method(*args, **kw)
  File "/home/dihnen/tmp/buildbot/sandbox/local/lib/python2.7/site-packages/buildbot_slave-0.8.5-py2.7.egg/buildslave/bot.py", line 141, in remote_startCommand
    d = self.command.doStart()
  File "/home/dihnen/tmp/buildbot/sandbox/local/lib/python2.7/site-packages/buildbot_slave-0.8.5-py2.7.egg/buildslave/commands/base.py", line 145, in doStart
    d = defer.maybeDeferred(self.start)

comment:3 Changed 15 months ago by DavidIAm

--- <exception caught here> ---
  File "/home/dihnen/tmp/buildbot/sandbox/local/lib/python2.7/site-packages/Twisted-11.1.0-py2.7-linux-x86_64.egg/twisted/internet/defer.py", line 134, in maybeDeferred
    result = f(*args, **kw)
  File "/home/dihnen/tmp/buildbot/sandbox/local/lib/python2.7/site-packages/buildbot_slave-0.8.5-py2.7.egg/buildslave/commands/shell.py", line 41, in start
    logEnviron=args.get('logEnviron', True),
Last edited 15 months ago by dustin (previous) (diff)

comment:4 Changed 15 months ago by DavidIAm

File "/home/dihnen/tmp/buildbot/sandbox/local/lib/python2.7/site-packages/buildbot_slave-0.8.5-py2.7.egg/buildslave/runprocess.py", line 307, in init

newenv[key] = p.sub(subst, environ[key])

exceptions.TypeError?: expected string or buffer exceptions.TypeError?: expected string or buffer ]

comment:5 Changed 15 months ago by tom.prince

  • Status changed from new to accepted
  • Priority changed from major to minor
  • Owner set to tom.prince
  • Version changed from 0.8.5 to master
  • Milestone changed from undecided to 0.8.+
  • Keywords property, removed

What is going on here is that Property("buildnumber") gets rendered as an int, which then causes the environment substitution to fail later.

Certainly we should have a better error message here, but I'm not sure if we want to be automatically coercing things to strings. Certainly it is reasonable for ints, but we don't also want to do it for arbitrary objects.

A work around is to do WithProperty("%(buildnumber)s"), which does deliberately coerce things.

comment:6 Changed 15 months ago by DavidIAm

Update documentation at the very least to make it clear that Property doesn't change the type of the variable, and that environment variable values must be strings. Probably also be a goo d idea to annotate the data type of each of the common properties listed.

comment:7 Changed 15 months ago by Dustin J. Mitchell

  • Status changed from accepted to closed
  • Resolution set to fixed

Only accept strings/lists for env values

This includes a docs update as well as a better exception. Fixes #2212.

Changeset: a1c7b124aead9d866fae881f5280266c0a085ef5

Note: See TracTickets for help on using tickets.