Opened 19 months ago

Last modified 12 months ago

#3593 new defect

GitPoller crashes when the repository has too many branches (MySQL Truncates JSON).

Reported by: rayene Owned by:
Priority: critical Milestone: 0.9.+
Version: 0.9.0b9 Keywords: db
Cc:

Description (last modified by dustin)

GitPoller? stores a json value in the object_state MySQL table. The type of value_json field is TEXT. TEXT is limited to 65,535 bytes.

When many branches with long names exist in the git repo, the json string generated by GitPoller? can be larger than 65,535 bytes. MySQL truncates the string.

When GitPoller? tries to read back the value, it does not find a well-formed json string.

Exception :

2016-08-05 18:33:49+0000 [-] Got fatal Exception on DB
	Traceback (most recent call last):
	Failure: exceptions.TypeError: JSON error loading state value 'lastRev' for 2
	
2016-08-05 18:33:49+0000 [-] Starting factory <twisted.web.client._HTTP11ClientFactory instance at 0x7f5bbd3f2cb0>
2016-08-05 18:33:49+0000 [-] while initializing GitPoller repository
	Traceback (most recent call last):
	  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 434, in errback
	    self._startRunCallbacks(fail)
	  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 501, in _startRunCallbacks
	    self._runCallbacks()
	  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 588, in _runCallbacks
	    current.result = callback(current.result, *args, **kw)
	  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1184, in gotResult
	    _inlineCallbacks(r, g, deferred)
	--- <exception caught here> ---
	  File "/usr/local/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1126, in _inlineCallbacks
	    result = result.throwExceptionIntoGenerator(g)
	  File "/usr/local/lib/python2.7/dist-packages/twisted/python/failure.py", line 389, in throwExceptionIntoGenerator
	    return g.throw(self.type, self.value, self.tb)
	  File "/usr/local/lib/python2.7/dist-packages/buildbot/util/state.py", line 31, in getState
	    **kwargs)
	  File "/usr/local/lib/python2.7/dist-packages/twisted/python/threadpool.py", line 246, in inContext
	    result = inContext.theWork()
	  File "/usr/local/lib/python2.7/dist-packages/twisted/python/threadpool.py", line 262, in <lambda>
	    inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
	  File "/usr/local/lib/python2.7/dist-packages/twisted/python/context.py", line 118, in callWithContext
	    return self.currentContext().callWithContext(ctx, func, *args, **kw)
	  File "/usr/local/lib/python2.7/dist-packages/twisted/python/context.py", line 81, in callWithContext
	    return func(*args,**kw)
	  File "/usr/local/lib/python2.7/dist-packages/buildbot/db/pool.py", line 175, in __thd
	    rv = callable(arg, *args, **kwargs)
	  File "/usr/local/lib/python2.7/dist-packages/buildbot/db/state.py", line 93, in thd
	    return self.thdGetState(conn, objectid, name, default=default)
	  File "/usr/local/lib/python2.7/dist-packages/buildbot/db/state.py", line 115, in thdGetState
	    (name, objectid))
	exceptions.TypeError: JSON error loading state value 'lastRev' for 2

Change History (3)

comment:1 Changed 19 months ago by rayene

Workaround :

ALTER TABLE object_state MODIFY value_json MEDIUMTEXT;

comment:2 Changed 19 months ago by dustin

  • Description modified (diff)
  • Keywords db added
  • Milestone changed from undecided to 0.9.+
Note: See TracTickets for help on using tickets.