Opened 4 years ago

Last modified 4 years ago

#2600 new defect

buildbot try doesn't work with patches >=64kb when using a mysql database

Reported by: dluntzer Owned by:
Priority: major Milestone: 0.9.+
Version: 0.8.8 Keywords: database try simple
Cc:

Description

When using pb as connection method for try builds the patches are storred in the database. The current buildbot implementation uses the mysql-type "text" to store the patch-data. According to the mysql-documentation the maximum size of this type is 64kb (http://dev.mysql.com/doc/refman/5.0/en/storage-requirements.html).

When submitting a try-build with a patch-size larger than 64kb the try command does not return an error message but the patch is never processed. According to the output of the web interface (see output below) base64 decoding of the patch fails and the triggered buildslaves stop working.

To solve this exception the patch has to be manually removed from the database.

If the 64kb limit for mysql is intended it would be better to not accept patches exceeding this limit and to return an error message. Otherwise changing the datatype to mediumtext (limit 16mb) or longtext could solve this issue...

web.Server Traceback (most recent call last): exceptions.TypeError?: Incorrect padding /usr/local/lib/python2.7/dist-packages/Twisted-13.2.0-py2.7-linux-i686.egg/twisted/internet/defer.py:1097 in _inlineCallbacks 1096 if isFailure: 1097 result = result.throwExceptionIntoGenerator(g) 1098 else: /usr/local/lib/python2.7/dist-packages/Twisted-13.2.0-py2.7-linux-i686.egg/twisted/python/failure.py:389 in throwExceptionIntoGenerator 388 """ 389 return g.throw(self.type, self.value, self.tb) 390 /usr/local/lib/python2.7/dist-packages/buildbot-0.8.8-py2.7.egg/buildbot/status/web/builder.py:266 in content 265 266 source = yield pb.getSourceStamp() 267 submitTime = yield pb.getSubmitTime() /usr/local/lib/python2.7/dist-packages/Twisted-13.2.0-py2.7-linux-i686.egg/twisted/internet/defer.py:1097 in _inlineCallbacks 1096 if isFailure: 1097 result = result.throwExceptionIntoGenerator(g) 1098 else: /usr/local/lib/python2.7/dist-packages/Twisted-13.2.0-py2.7-linux-i686.egg/twisted/python/failure.py:389 in throwExceptionIntoGenerator 388 """ 389 return g.throw(self.type, self.value, self.tb) 390 /usr/local/lib/python2.7/dist-packages/buildbot-0.8.8-py2.7.egg/buildbot/status/buildrequest.py:85 in getSourceStamp 84 def getSourceStamp(self): 85 br = yield self._getBuildRequest() 86 defer.returnValue(br.source) /usr/local/lib/python2.7/dist-packages/Twisted-13.2.0-py2.7-linux-i686.egg/twisted/internet/defer.py:1097 in _inlineCallbacks 1096 if isFailure: 1097 result = result.throwExceptionIntoGenerator(g) 1098 else: /usr/local/lib/python2.7/dist-packages/Twisted-13.2.0-py2.7-linux-i686.egg/twisted/python/failure.py:389 in throwExceptionIntoGenerator 388 """ 389 return g.throw(self.type, self.value, self.tb) 390 /usr/local/lib/python2.7/dist-packages/buildbot-0.8.8-py2.7.egg/buildbot/status/buildrequest.py:58 in _getBuildRequest 57 br = yield buildrequest.BuildRequest?.fromBrdict(self.master, 58 brd) 59 self._buildrequest = br /usr/local/lib/python2.7/dist-packages/Twisted-13.2.0-py2.7-linux-i686.egg/twisted/internet/defer.py:1097 in _inlineCallbacks 1096 if isFailure: 1097 result = result.throwExceptionIntoGenerator(g) 1098 else: /usr/local/lib/python2.7/dist-packages/Twisted-13.2.0-py2.7-linux-i686.egg/twisted/python/failure.py:389 in throwExceptionIntoGenerator 388 """ 389 return g.throw(self.type, self.value, self.tb) 390 /usr/local/lib/python2.7/dist-packages/buildbot-0.8.8-py2.7.egg/buildbot/process/buildrequest.py:116 in _make_br 115 # fetch the sourcestamp dictionary 116 sslist = yield master.db.sourcestamps.getSourceStamps(buildsetsourcestampsetid?) 117 assert len(sslist) > 0, "Empty sourcestampset: db schema enforces set to exist but cannot enforce a non empty set" /usr/local/lib/python2.7/dist-packages/Twisted-13.2.0-py2.7-linux-i686.egg/twisted/internet/defer.py:1097 in _inlineCallbacks 1096 if isFailure: 1097 result = result.throwExceptionIntoGenerator(g) 1098 else: /usr/local/lib/python2.7/dist-packages/Twisted-13.2.0-py2.7-linux-i686.egg/twisted/python/failure.py:389 in throwExceptionIntoGenerator 388 """ 389 return g.throw(self.type, self.value, self.tb) 390 /usr/local/lib/python2.7/dist-packages/buildbot-0.8.8-py2.7.egg/buildbot/db/sourcestamps.py:95 in getSourceStamps 94 for ssid in ssids: 95 sourcestamp = yield self.getSourceStamp(ssid) 96 sslist.append(sourcestamp) /usr/local/lib/python2.7/dist-packages/Twisted-13.2.0-py2.7-linux-i686.egg/twisted/python/threadpool.py:191 in _worker 190 try: 191 result = context.call(ctx, function, *args, kwargs) 192 success = True /usr/local/lib/python2.7/dist-packages/Twisted-13.2.0-py2.7-linux-i686.egg/twisted/python/context.py:118 in callWithContext 117 def callWithContext(self, ctx, func, *args, kw): 118 return self.currentContext().callWithContext(ctx, func, *args, kw) 119 /usr/local/lib/python2.7/dist-packages/Twisted-13.2.0-py2.7-linux-i686.egg/twisted/python/context.py:81 in callWithContext 80 try: 81 return func(*args,kw) 82 finally: /usr/local/lib/python2.7/dist-packages/buildbot-0.8.8-py2.7.egg/buildbot/db/pool.py:184 in thd 183 try: 184 rv = callable(arg, *args, kwargs) 185 assert not isinstance(rv, sa.engine.ResultProxy?), \ /usr/local/lib/python2.7/dist-packages/buildbot-0.8.8-py2.7.egg/buildbot/db/sourcestamps.py:129 in thd 128 ssdictpatch_comment? = row.patch_comment 129 body = base64.b64decode(row.patch_base64) 130 ssdictpatch_body? = body /usr/lib/python2.7/base64.py:76 in b64decode 75 # Transform this exception for consistency 76 raise TypeError?(msg) 77 exceptions.TypeError?: Incorrect padding

Attachments (2)

output.html (19.1 KB) - added by dluntzer 4 years ago.
output.2.html (19.1 KB) - added by dluntzer 4 years ago.

Download all attachments as: .zip

Change History (5)

Changed 4 years ago by dluntzer

Changed 4 years ago by dluntzer

comment:1 Changed 4 years ago by dluntzer

  • Summary changed from buildbot try doesn't work with patches >=64kb when using a mysqlite database to buildbot try doesn't work with patches >=64kb when using a mysql database

comment:2 Changed 4 years ago by dustin

  • Keywords simple added; mysql removed
  • Milestone changed from undecided to 0.9.0

Agreed - we should probably increase the column size. DB changes are frozen in 0.8.x, so that would need to happen in the nine branch.

comment:3 Changed 4 years ago by dustin

  • Milestone changed from 0.9.0 to 0.9.+
Note: See TracTickets for help on using tickets.