Crash when handling a lot of objects

[Almad]

Using git post-push hook, we get following:

Unhandled error in Deferred:
Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 285, in unpause
    self._runCallbacks()
  File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 328, in _runCallbacks
    self.result = callback(self.result, *args, **kw)
  File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 289, in _continue
    self.unpause()
  File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 285, in unpause
    self._runCallbacks()
--- <exception caught here> ---
  File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 328, in _runCallbacks
    self.result = callback(self.result, *args, **kw)
  File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 289, in _continue
    self.unpause()
  File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 285, in unpause
    self._runCallbacks()
  File "/usr/lib/python2.5/site-packages/twisted/internet/defer.py", line 344, in _runCallbacks
    self.result = failure.Failure()
  File "/usr/lib/python2.5/site-packages/twisted/python/failure.py", line 265, in __init__
    parentCs = reflect.allYourBase(self.type)
  File "/usr/lib/python2.5/site-packages/twisted/python/reflect.py", line 542, in allYourBase
    accumulateBases(classObj, l, baseClass)
  File "/usr/lib/python2.5/site-packages/twisted/python/reflect.py", line 550, in accumulateBases
    accumulateBases(base, l, baseClass)
  File "/usr/lib/python2.5/site-packages/twisted/python/reflect.py", line 550, in accumulateBases
    accumulateBases(base, l, baseClass)
  File "/usr/lib/python2.5/site-packages/twisted/python/reflect.py", line 550, in accumulateBases
    accumulateBases(base, l, baseClass)
  File "/usr/lib/python2.5/site-packages/twisted/python/reflect.py", line 550, in accumulateBases
    accumulateBases(base, l, baseClass)
exceptions.RuntimeError: maximum recursion depth exceeded

This seems to happen when a lot of objects is being pushed.

[dustin]

is this in git_buildbot.py?

[Almad]

Yes, it is.

Furter discussion on IRC:

[15:46] <exarkun> might be http://twistedmatrix.com/trac/ticket/411
[15:46] <exarkun> though it's not obvious to me how you'd end up with that many Deferreds when handling a new revision

[15:55] <djmitche> exarkun: addChange is exactly the second kind of loop warner discussed in the allmydata bug
[15:55] <djmitche> it's "tail recursive"
[15:55] <djmitche> Almad: I suspect that changing
[15:55] <djmitche>  76     d.addCallback(addChange, remote, changei)
[15:56] <djmitche> to instead use reactor.callLater(addChange, None, remote, changei) would fix this


def addChange(dummy, remote, changei):
    logging.debug("addChange %s, %s" % (repr(remote), repr(changei)))
    try:
        c = changei.next()
    except StopIteration:
        remote.broker.transport.loseConnection()
        return None

    logging.info("New revision: %s" % c['revision'][:8])
    for key, value in c.iteritems():
        logging.debug("  %s: %s" % (key, value))

    d = remote.callRemote('addChange', c)
    def recurseLater(x):
        reactor.callLater(0, addChange(x, remote, changei)
    d.addCallback(recurseLater)
    return d

[dustin]

Did this suggestion fix the problem?

[dustin]

A (hopefully more correct) variant:

commit fb70a1dd799ff16ef9b3002707ed3613c5780c0d
Author: Dustin J. Mitchell <dustin@zmanda.com>
Date:   Sat Dec 19 09:52:55 2009 -0600

    (fixes #645) remove tail recursion

Please reopen if this isn't fixed.