Opened 6 years ago

Closed 6 years ago

Last modified 6 years ago

#3112 closed defect (fixed)

Git step fails when source dir is missing

Reported by: dustin Owned by: dustin
Priority: critical Milestone: 0.8.x
Version: 0.8.10 Keywords: git
Cc:

Description

On the slave:

2014-12-14 17:23:09+0000 [Broker,client] startCommand:listdir [id 7] 2014-12-14 17:23:09+0000 [Broker,client] ListDir /slave/build-cpp-ethereum-pr/source failed [Errno 2] No such file or directory: '/slave/build-cpp-ethereum-pr/source' 2014-12-14 17:23:09+0000 [Broker,client] SlaveBuilder.commandComplete <buildslave.commands.fs.ListDir instance at 0x7f33d437cef0>

On the master:

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 423, in errback
    self._startRunCallbacks(fail)
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 490, in _startRunCallbacks
    self._runCallbacks()
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 577, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1155, in gotResult
    _inlineCallbacks(r, g, deferred)
--- <exception caught here> ---
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1097, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/home/buildbot/buildbot/master/buildbot/steps/source/git.py", line 184, in full
    yield self.copy()
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1097, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/local/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/home/buildbot/buildbot/master/buildbot/steps/source/git.py", line 203, in incremental
    action = yield self._sourcedirIsUpdatable()
  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 577, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/home/buildbot/buildbot/master/buildbot/steps/source/git.py", line 580, in checkWithListdir
    files = cmd.updates['files'][0]
exceptions.KeyError: 'files'

which isn't surprising - when the directory doesn't exist, listdir doesn't return files.

Change History (10)

comment:1 Changed 6 years ago by sa2ajj

Per user's comment it works with 0.8.9

comment:2 Changed 6 years ago by sa2ajj

  • Keywords git added

comment:3 Changed 6 years ago by Dustin J. Mitchell <dustin@…>

comment:4 Changed 6 years ago by Dustin J. Mitchell <dustin@…>

  • Resolution set to fixed
  • Status changed from new to closed

In 6d82171683031b8847975daa2b1408693ad3aaeb:

when listdir returns no files, clone

Fixes #3112.

comment:5 Changed 6 years ago by Dustin J. Mitchell <dustin@…>

In 7b9fb2d23e9ea07e1757fb31b5c309ab4e51bbf7:

Merge djmitche/buildbot:bug3112 (PR #1447)

comment:6 Changed 6 years ago by yarikoptic

I am observing the same issue with the current master (v0.9.0-pre-1265-gcb9b4c0). Started to appear after my initial attempt to follow http://github.com/ethereum/ethereum-buildbot setup to provide testing of the pull requests. Here is my changes: https://github.com/datalad/buildbot/commit/455e2049e3ce6435907ac00e737d4e25b5190f6d

Here is backtrace for completeness:

...
          self.results = yield self.run()
          File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1099, in _inlineCallbacks
            result = g.send(result)
          File "/home/buildbot/buildbot/venv/local/lib/python2.7/site-packages/buildbot/process/buildstep.py", line 602, in run
            results = yield self._start_deferred
          File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1097, in _inlineCallbacks
            result = result.throwExceptionIntoGenerator(g)
          File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
            return g.throw(self.type, self.value, self.tb)
          File "/home/buildbot/buildbot/venv/local/lib/python2.7/site-packages/buildbot/steps/source/git.py", line 185, in full
            yield self.copy()
          File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1097, in _inlineCallbacks
            result = result.throwExceptionIntoGenerator(g)
          File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
            return g.throw(self.type, self.value, self.tb)
          File "/home/buildbot/buildbot/venv/local/lib/python2.7/site-packages/buildbot/steps/source/git.py", line 204, in incremental
            action = yield self._sourcedirIsUpdatable()
          File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 577, in _runCallbacks
            current.result = callback(current.result, *args, **kw)
          File "/home/buildbot/buildbot/venv/local/lib/python2.7/site-packages/buildbot/steps/source/git.py", line 594, in checkWithListdir
            files = cmd.updates['files'][0]
        exceptions.KeyError: 'files'

I had mode='copy'... commenting it out it started to work. so it might be worth may be sophisticating checks even further for such silly oes like me overriding logic with custom copy without having a local copy?

comment:7 Changed 6 years ago by dustin

yarikoptic, I'm not sure what you mean -- did you customize the step somehow? Can you create a test case or a reduced master configuration that will reproduce this?

comment:8 Changed 6 years ago by yarikoptic

I have pointed to all "customizations" I have done at that point ie https://github.com/datalad/buildbot/commit/455e2049e3ce6435907ac00e737d4e25b5190f6d#diff-df27f81b607a8a677c3117b98088abaeR89 and using it in a similar shape since then (without mode='copy') but now based on "eight" branch

comment:9 Changed 6 years ago by dustin

Ah! It looks like the commit you named, [7b9fb2d23e9ea07e1757fb31b5c309ab4e51bbf7], wasn't forward-ported. I'll fix that up now.

comment:10 Changed 6 years ago by dustin

Note: See TracTickets for help on using tickets.