buildbot is unable to delete subdirectories of the build dir that have non-ASCII names under Windows

[davidsarah]

One of the unit tests in Tahoe-LAFS creates a subdirectory that has a non-ASCII name (locāl_dir). On Windows, this causes an error on subsequent builds when buildbot tries to delete the "workdir" for the build. This seems to happen because it ASCIIfies the name to "local_dir" when trying to change its permissions to ensure it is deleteable.

The traceback is:

starting darcs operation

remoteFailed: [Failure instance: Traceback from remote host -- Traceback (most recent call last):
  File "c:python27libsite-packagesuildbot-0.7.12-py2.7.egguildbotslavecommands.py", line 1425, in start
    self.maybeClobber(d)
  File "c:python27libsite-packagesuildbot-0.7.12-py2.7.egguildbotslavecommands.py", line 1444, in maybeClobber
    d.addCallback(self.doClobber, self.workdir)
  File "c:python27libsite-packages	wistedinternetdefer.py", line 297, in addCallback
    callbackKeywords=kw)
  File "c:python27libsite-packages	wistedinternetdefer.py", line 286, in addCallbacks
    self._runCallbacks()
--- <exception caught here> ---
  File "c:python27libsite-packages	wistedinternetdefer.py", line 542, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "c:python27libsite-packagesuildbot-0.7.12-py2.7.egguildbotslavecommands.py", line 1600, in doClobber
    rmdirRecursive(d)
  File "c:python27libsite-packagesuildbot-0.7.12-py2.7.egguildbotslavecommands.py", line 127, in rmdirRecursive
    rmdirRecursive(full_name)
  File "c:python27libsite-packagesuildbot-0.7.12-py2.7.egguildbotslavecommands.py", line 127, in rmdirRecursive
    rmdirRecursive(full_name)
  File "c:python27libsite-packagesuildbot-0.7.12-py2.7.egguildbotslavecommands.py", line 124, in rmdirRecursive
    os.chmod(full_name, 0600)
exceptions.WindowsError: [Error 2] Le fichier spécifié est introuvable: 'C:\buildbot_tahoe\FreeStorm_WinXP-x86_py2.6\build\_trial_temp_old228388\drop_upload.MockTest.test_drop_upload\local_dir'
]

An example of code that more reliably removes a directory tree is the rm_dir function in  http://tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/src/allmydata/util/fileutil.py.

This corresponds to  Tahoe-LAFS ticket 1472.

[dustin]

Can you provide a patch for Buildbot that uses such code? Even better would be such a patch along with a test!

[davidsarah]

Yes, I can. Since the immediate problem with Tahoe-LAFS has been worked around, it'll have to wait for a few weeks until after our release.

[dustin]

Any progress here?

[davidsarah]

Sorry, this got pushed down my priority list when I got a new job. Will try to get to it in the next two weeks, unless someone else wants to take it over.

[dustin]

I expect that it's all yours :)