Opened 2 years ago

Last modified 13 months ago

#3384 new defect

UnicodeDecodeError upon finishing BuildStep run

Reported by: jollyroger Owned by:
Priority: major Milestone: undecided
Version: 0.8.12 Keywords: buildstep
Cc:

Description

Running npm install inside a build step may cause a sometimes weird output appear like this:

⚠ pre-build test failed, compiling from source...
✔ pre-build test passed successfully!
✔ pre-build test passed successfully!

> optipng-bin@1.0.1 postinstall /home/broq/buildbot/karkinos/rpmbuild/build/BUILD/broq-0.5.2.4-1-g2cf5752/build/www/node_modules/grunt-contrib-imagemin/node_modules/imagemin/node_modules/imagemin-optipng/node_modules/optipng-bin
> node lib/install.js

✔ pngquant built successfully!
⚠ pre-build test failed, compiling from source...
✔ optipng built successfully!

While this output was absolutely ok for 0.8.9, it causes buildstep to fail with an exception on the first character for 0.8.12:

2015-11-16 20:37:46+0000 [-] BuildStep.failed; traceback follows
	Traceback (most recent call last):
	  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1101, in gotResult
	    _inlineCallbacks(r, g, deferred)
	  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1088, in _inlineCallbacks
	    deferred.callback(e.value)
	  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 368, in callback
	    self._startRunCallbacks(result)
	  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 464, in _startRunCallbacks
	    self._runCallbacks()
	--- <exception caught here> ---
	  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 551, in _runCallbacks
	    current.result = callback(current.result, *args, **kw)
	  File "/usr/lib/python2.7/dist-packages/buildbot/process/buildstep.py", line 650, in <lambda>
	    d.addCallback(lambda res: self.commandComplete(cmd))
	  File "/usr/lib/python2.7/dist-packages/buildbot/steps/shell.py", line 337, in commandComplete
	    self.observer.getStdout(),
	  File "/usr/lib/python2.7/dist-packages/buildbot/process/logobserver.py", line 127, in getStdout
	    return self._get(self.stdout)
	  File "/usr/lib/python2.7/dist-packages/buildbot/process/logobserver.py", line 119, in _get
	    result = u''.join(chunks)
	exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 512: ordinal not in range(128)
	
2015-11-16 20:37:46+0000 [-] addCompleteLog(err.text)
2015-11-16 20:37:46+0000 [-] addHTMLLog(err.html)
2015-11-16 20:37:46+0000 [-] releaseLocks(<broqbot.steps.RpmBuild object at 0xae78a40c>): []
2015-11-16 20:37:46+0000 [-] BuildStep.failed now firing callback
2015-11-16 20:37:46+0000 [-]  step 'rpmbuild' complete: exception
2015-11-16 20:37:46+0000 [-]  <Build rpmbuild>: build finished
2015-11-16 20:37:46+0000 [-] Starting factory <twisted.mail.smtp.ESMTPSenderFactory instance at 0xaea62fec>
2015-11-16 20:37:46+0000 [-] pruning '/var/lib/broq/master/rpmbuild/1396'
2015-11-16 20:37:46+0000 [-] pruning '/var/lib/broq/master/rpmbuild/1396-log-shell-stdio.bz2'
2015-11-16 20:37:46+0000 [-] pruning '/var/lib/broq/master/rpmbuild/1396-log-rpmbuild-property_changes'
2015-11-16 20:37:46+0000 [-] pruning '/var/lib/broq/master/rpmbuild/1396-log-rpmbuild-stdio.bz2'
2015-11-16 20:37:46+0000 [-] releaseLocks(<BuildSlave 'karkinos'>): []

Here is a one such exception in more detail:

UnicodeDecodeError('ascii', '\n> jpegtran-bin@1.0.2 postinstall /home/broq/buildbot/karkinos/rpmbuild/build/BUILD/broq-0.5.2.4-1-g2cf5752/build/www/node_modules/grunt-contrib-imagemin/node_modules/imagemin/node_modules/imagemin-jpegtran/node_modules/jpegtran-bin\n> node lib/install.js\n\n\xe2\x9a\xa0 pre-build test failed, compiling from source...\n\xe2\x9c\x94 pre-build test passed successfully!\n\xe2\x9c\x94 pre-build test passed successfully!\n\n> optipng-bin@1.0.1 postinstall /home/broq/buildbot/karkinos/rpmbuild/build/BUILD/broq-0.5.2.4-1-g2cf5752/build/www/node_modules/grunt-contrib-imagemin/node_modules/imagemin/node_modules/imagemin-optipng/node_modules/optipng-bin\n> node lib/install.js\n\n', 256, 257, 'ordinal not in range(128)')

Don't know if this is interesting, but the mentioned text appears in the middle of the build step run, although exception is thrown when the whole build step is finishing.

Change History (2)

comment:1 Changed 2 years ago by jollyroger

A possible fix here (at least in my case):

/usr/lib/python2.7/dist-packages/buildbot/process/logobserver.py

    def _get(self, chunks):
        if chunks is None or not chunks:
            return u'' 
        return ''.join(chunks).decode('utf8')

comment:2 Changed 13 months ago by RockHed

Just hit this too, applying the same patch. This throws when the build step finishes because that's when the SetProperty? runs - I am assuming you are using SetPropertyFromCommand? or similar.

Note: See TracTickets for help on using tickets.