Opened 10 years ago

# Slave's ShellCommand and msys are broken

Reported by: Owned by: even even major 0.9.+ 0.7.10 windows fqueze

### Description

While using buildbot in msys, new bugs appeared somewhere between 0.7.9 and 0.7.10p1.

Those are related to the changes in the ShellCommand? object even if, right now, I can't tell what exactly caused this. What I know is, when I call pwd using buildbot, it used to return a path in msys path format (full UNIX like, so it was looking alike /c/some/folder) when it returns a path in windows format now (c:path ofolder). Using some tricks with environment variables before launching buildbot I achieved to get UNIX compatible path using the windows format (like c:\path\to\folder) but it stills cause problems on some executables like make that completely fails.

Here is what the error looks like :

make -f client.mk build
in dir c:uildbotxp-compilwin32uild (timeout 1200 secs)
watching logfiles {}
argv: ['make', '-f', 'client.mk', 'build']
environment:
!EXITCODE=00000001
ALLUSERSPROFILE=C:Documents and SettingsAll Users
COMMONPROGRAMFILES=C:Program FilesFichiers communs
COMPUTERNAME=XP-32-COMPIL
COMSPEC=C:WINDOWSsystem32cmd.exe
CVS_RSH=ssh
DEVENVDIR=C:Program FilesMicrosoft Visual Studio 8Common7IDE
EDITOR=xemacs.exe
FP_NO_HOST_CHECK=NO
FRAMEWORKDIR=C:WINDOWSMicrosoft.NETFramework
FRAMEWORKSDKDIR=C:Program FilesMicrosoft Visual Studio 8SDKv2.0
FRAMEWORKVERSION=v2.0.50727
HISTFILE=C:/mozilla-build/msys/home/buildbot/.bash_history
HOME=c:/Documents and Settings/buildbot
HOMEDRIVE=C:
HOMEPATH=
HOSTNAME=XP-32-COMPIL
HOSTTYPE=i686
INCLUDE=C:Program FilesMicrosoft SDKsWindowsv6.0\include;C:Program FilesMicrosoft SDKsWindowsv6.0\includeatl;C:Program FilesMicrosoft Visual Studio 8VCATLMFCINCLUDE;C:Program FilesMicrosoft Visual Studio 8VCINCLUDE;C:Program FilesMicrosoft Visual Studio 8VCPlatformSDKinclude;C:Program FilesMicrosoft Visual Studio 8SDKv2.0include;
INPUTRC=C:/mozilla-build/msys/etc/inputrc
LIB=C:Program FilesMicrosoft SDKsWindowsv6.0\lib;C:Program FilesMicrosoft Visual Studio 8VCATLMFCLIB;C:Program FilesMicrosoft Visual Studio 8VCLIB;C:Program FilesMicrosoft Visual Studio 8VCPlatformSDKlib;C:Program FilesMicrosoft Visual Studio 8SDKv2.0lib;;C:mozilla-buildatlthunk_compat
LIBPATH=C:WINDOWSMicrosoft.NETFrameworkv2.0.50727;C:Program FilesMicrosoft Visual Studio 8VCATLMFCLIB
LOGNAME=buildbot
LOGONSERVER=\XP-32-COMPIL
MACHTYPE=i686-pc-msys
MAKEFLAGS= --unix
MAKE_MODE=unix
MFLAGS=- --unix
MOZBUILDDIR=C:mozilla-build
MOZCONFIG=c:uildbotxp-compilwin32uild/tools/buildbot-configs/win32/mozconfig
MOZILLABUILD=C:mozilla-build
MOZ_MSVCVERSION=8
MOZ_TOOLS=C:mozilla-buildmoztools
MSVC6KEY=HKLMSOFTWAREMicrosoftVisualStudio6.0SetupMicrosoft Visual C++
MSVC71KEY=HKLMSOFTWAREMicrosoftVisualStudio7.1SetupVC
MSVC8EXPRESSKEY=HKLMSOFTWAREMicrosoftVCExpress8.0SetupVC
MSVC8KEY=HKLMSOFTWAREMicrosoftVisualStudio8.0SetupVC
MSVC9EXPRESSKEY=HKLMSOFTWAREMicrosoftVCExpress9.0SetupVC
MSVC9KEY=HKLMSOFTWAREMicrosoftVisualStudio9.0SetupVC
MSVCROOTKEY=HKLMSOFTWAREMicrosoftVisualStudio
MSYSTEM=MINGW32
NUMBER_OF_PROCESSORS=1
OLDPWD=c:/mozilla-build/python25/lib/site-packages/win32
OLD_PATH=C:Program FilesWindows Resource KitsTools;C:WINDOWSsystem32;C:WINDOWS;C:WINDOWSSystem32Wbem;c:Program FilesMicrosoft SQL Server90Toolsinn;C:mozilla-buildPython25;C:mozilla-buildPython25Scripts
OS=Windows_NT
OSTYPE=msys
PATH=C:mozilla-buildmsyslocalin;c:mozilla-buildwget;c:mozilla-build7zip;c:mozilla-buildlat261full;c:mozilla-buildpython25;c:mozilla-buildsvn-win32-1.4.2in;c:mozilla-buildupx203w;c:mozilla-buildxemacsXEmacs-21.4.19i586-pc-win32;c:mozilla-buildinfo-zip;c:mozilla-build
sis-2.22;c:mozilla-build
sis-2.33u;c:mozilla-buildhg;c:mozilla-buildpython25Scripts;.;C:mozilla-buildmsyslocalin;C:mozilla-buildmsysmingwin;C:mozilla-buildmsysin;c:Program FilesMicrosoft SDKsWindowsv6.0in;c:Program FilesMicrosoft Visual Studio 8Common7IDE;c:Program FilesMicrosoft Visual Studio 8VCBIN;c:Program FilesMicrosoft Visual Studio 8Common7Tools;c:Program FilesMicrosoft Visual Studio 8Common7Toolsin;c:Program FilesMicrosoft Visual Studio 8VCPlatformSDKin;c:Program FilesMicrosoft Visual Studio 8SDKv2.0in;c:WINDOWSMicrosoft.NETFrameworkv2.0.50727;c:Program FilesMicrosoft Visual Studio 8VCVCPackages;c:WINDOWSSystem32;c:WINDOWS;c:WINDOWSSystem32Wbem;c:mozilla-buildmoztoolsin
PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.PY
PROCESSOR_ARCHITECTURE=x86
PROCESSOR_IDENTIFIER=x86 Family 6 Model 23 Stepping 7, GenuineIntel
PROCESSOR_LEVEL=6
PROCESSOR_REVISION=1707
PROGRAMFILES=C:Program Files
PROMPT=$P$G
PS1=[33]0;$MSYSTEM:w07 33[32m]u@h [33[33mw33[0m]$
PWD=c:uildbotxp-compilwin32uild
PYTHON=python
SDK2003SP1KEY=HKLMSOFTWAREMicrosoftMicrosoftSDKInstalledSDKs8F9E5EF3-A9A5-491B-A889-C58EFFECE8B3
SDK2003SP2KEY=HKLMSOFTWAREMicrosoftMicrosoftSDKInstalledSDKsD2FF9F89-8AA2-4373-8A31-C838BF4DBBE1
SDK6KEY=HKLMSOFTWAREMicrosoftMicrosoft SDKsWindowsv6.0WinSDKBuild
SDKDIR=C:Program FilesMicrosoft SDKsWindowsv6.0
SDKMINORVER=0
SDKROOTKEY=HKLMSOFTWAREMicrosoftMicrosoftSDKInstalledSDKs
SDKVER=6
SHELL=C:/mozilla-build/msys/bin/sh
SHLVL=1
SSH_AGENT_PID=3232
SSH_AUTH_SOCK=C:/DOCUME~1/buildbot/LOCALS~1/Temp/ssh-vHocOE3172/agent.3172
SYSTEMDRIVE=C:
SYSTEMROOT=C:WINDOWS
TEMP=C:/DOCUME~1/buildbot/LOCALS~1/Temp
TERM=cygwin
TMP=C:/DOCUME~1/buildbot/LOCALS~1/Temp
USERDOMAIN=XP-32-COMPIL
USERPROFILE=C:Documents and Settingsuildbot
USESDK=1
VC8DIR=C:Program FilesMicrosoft Visual Studio 8VC
VCINSTALLDIR=C:Program FilesMicrosoft Visual Studio 8VC
VS80COMNTOOLS=C:Program FilesMicrosoft Visual Studio 8Common7Tools
VSINSTALLDIR=C:Program FilesMicrosoft Visual Studio 8
WINDIR=C:WINDOWS
_=c:/mozilla-build/python25/python
closing stdin
using PTY: False
/bin/sh: c:buildbotxp-compilwin32build/client.mk: No such file or directory
client.mk:118: *** This source tree appears to have Windows-style line endings. To convert it to Unix-style line endings, run "python mozilla/build/win32/mozilla-dos2unix.py".  Stop.
program finished with exit code 2
elapsedTime=0.563000


While looking at the main differences with the previous version I used (0.7.8) that was working, the call to cmd.exe that previously preceded each calls disappeared. That might be the reason, but also it might be something else.

What I know for sure is that I noticed the version of ShellCommand? I can find on this trac is not mine and if I replace mine with it (just adding a usePTY argument to the constructor to get it working), everything works again.

I intend to look further on this bug but don't have time right now. If someone happens to find the exact reason of this and to produce a patch quickly (meaning before I find more time to look about that), I would really appreciate it.

### comment:2 Changed 10 years ago by even

• Milestone changed from undecided to 0.7.11
• Owner set to even
• Status changed from new to assigned

I just added a patch correcting the bug on msys. It seems that changing the PWD variable is making everything bug on msys. It's better not to do it in these conditions.

If you ABSOLUTELY want to set PWD, I might try another patch that convert the path in msys format before setting it.

Since my patch does not affect other OS and seems to work, I add the review flag and set the milestone to next version.

Correct the bug.

### comment:4 Changed 10 years ago by blackett

I had the same problem with buildslaves in Windows Server 2008 that use cygwin

It seems that once the PWD was set with a windows like path, cygwin maintained this form, whereas I had scripts that ran assuming the cygwin form, which is what had been defined up to that point.

My workaround was to unset PWD in the bat file that starts my cygwin processes for buildbot:

  SET PWD=


### comment:5 Changed 10 years ago by dustin

• Resolution set to fixed
• Status changed from assigned to closed

### comment:6 Changed 9 years ago by hushp1pt

• Resolution fixed deleted
• Status changed from closed to reopened

I had the same problem with Cygwin-1.7 on a Windows 2003 server.

However, I too had the same pwd problem (Windows path when cygwin-style path was expected). I also tried to unset PWD, but I never got that to change anything.

I eventually commented out the

self.environPWD? = os.path.abspath(self.workdir)

statement in runprocess.py, and everything was great.

Can you add a test for cygwin as well as msys to the patch that fixed this bug for msys?

### comment:7 follow-up: ↓ 8 Changed 9 years ago by dustin

• Keywords review sprint removed

how can I identify a cygwin system in Python?

### comment:8 in reply to: ↑ 7 Changed 9 years ago by hushp1pt

how can I identify a cygwin system in Python?

I was wondering, in the earlier patch here, how did MACHTYPE know it was an MSYS system?

self.environ.get('MACHTYPE', None) == 'i686-pc-msys'

Do you need to start the buildslave from Msys for MACHTIME to say "msys"? If you started the buildslave from Cygwin, would MACHTYPE say "cygwin"?

In fact, I do not start our Windows buildslave from Cygwin. I used the run-buildbot-as-service contrib. But I still want the BuildCommand? commands to run under Cygwin.

If you don't have MACHTYPE that says Cygwin, there probably is no foolproof way to tell if a ShellCommand? buildstep wanted to run Cygwin. However, I bet if the first word of the first command in the build step was "bash", "bash.exe", or a full path to bash, and buildslave is Windows, then the odds would be pretty good that the command(s) were really meant for Cygwin bash- or for Msys bash. Either way, the patch would help.

Those who prefer to have their ShellCommand? start a BAT file to run Cygwin commands, I guess they can still set PWD= before they call bash or whatever.

### comment:9 follow-up: ↓ 10 Changed 9 years ago by dustin

• Milestone changed from 0.7.11 to 0.8.3

I'm worried about playing whack-a-mole here, or getting too specific in analyzing the components of a buildstep. Could we add a ShellCommand? option that would be passed to the slave to enable this particular behavior?

### comment:10 in reply to: ↑ 9 Changed 9 years ago by hushp1pt

I'm worried about playing whack-a-mole here, or getting too specific in analyzing the components of a buildstep. Could we add a ShellCommand? option that would be passed to the slave to enable this particular behavior?

Thanks, I think that would be ideal. You could call the option with_cygwin_path. If value == None, no change- same as regular Windows process. If != None, skip the PWD reset in runprocess.py.

For extra credit, if value looks like a Cygwin path string, then override the PATH that bash (or whatever) sees when it lands. In the example http://buildbot.net/trac/wiki/CygwinShellCommands, I passed Cygwin's PATH through Shell Command's envPATH?. Implementing some kind of short cut for this override would be helpful, because envPATH? was the only value in env that had to be different between Windows and Linux builds and I had serious python and/or buildbot language troubles in my build factory, trying to define that env dictionary the right way at the right time. Your call. Thanks again.

### comment:11 follow-up: ↓ 12 Changed 9 years ago by ayust

• Milestone changed from 0.8.3 to 0.8.+

### comment:12 in reply to: ↑ 11 Changed 7 years ago by hushp1pt

For extra credit, ...

Hi, please forget all about that "for extra credit" idea- my mistake.

Please reconsider the simple bug fix dustin proposed,

Could we add a ShellCommand option that would be passed to the slave to enable this particular behavior?

I suggest the new ShellCommand option be named with_cygwin_path. If with_cygwin_path == None, no change- same as regular Windows process. If != None, skip the PWD reset in runprocess.py.

Thanks.

### comment:13 Changed 7 years ago by tom.prince

I wonder if it actually makes sense to be setting PWD anywhere? If the command being run is a shell script, then the shell should be setting PWD. Otherwise, I'd guess that things should be using getcwd(3) to find the current directory, not a environment variable. Perhaps we should just unset it?

### comment:14 Changed 5 years ago by dustin

• Milestone changed from 0.8.+ to 0.9.+

Ticket retargeted after milestone closed

Note: See TracTickets for help on using tickets.