Re: [Condor-users] broken condor_exec.exe path on condor-g submit to windows pool

["Timothy St. Clair" <tstclair@xxxxxxxxxx>]

Inline below

On Sat, 2010-06-19 at 15:01 +0200, Rob de Graaf wrote:
> Hi Ben,
> 
> Thanks for the pointer, I didn't know about that option. Unfortunately, 
> it doesn't help in my case, as the startd now simply tries to execute 
> the non-existent path as binary:
> 
> ==============================================================
> 06/19 14:24:04 File transfer completed successfully.
> 06/19 14:24:05 Job 43.0 set to execute immediately
> 06/19 14:24:05 Starting a VANILLA universe job with ID: 43.0
> 06/19 14:24:05 Tracking process family by login "condor-reuse-slot1"
> 06/19 14:24:05 IWD: C:\Progra~1\Condor\execute\dir_716
> 06/19 14:24:05 Output file: 
> C:\Progra~1\Condor\execute\dir_716\_condor_stdout
> 06/19 14:24:05 Error file: C:\Progra~1\Condor\execute\dir_716\_condor_stderr
> 06/19 14:24:05 Renice expr "10" evaluated to 10
> 06/19 14:24:05 About to exec 
> C:\Progra~1\Condor\execute\dir_716\condor_exec.gass_cache/local/md5/6d/2254cb0171674e4280321100556c9a/md5/3f/ecb634a0691dc658e01762a1a0d474/data 
> 

This may be a shot in the dark, but I've seen numerous issues when
trying to run condor out of C:\Program Files\... because of the space in
the path.  The original install path defaults to C:\condor, you may want
to try giving that a whirl.  

> 06/19 14:24:05 ERROR: failed to produce Win32 argument string from 
> CreateProcess:
> 06/19 14:24:05 ERROR 
> "Create_Process(C:\Progra~1\Condor\execute\dir_716\condor_exec.gass_cache/local/md5/6d/2254cb0171674e4280321100556c9a/md5/3f/ecb634a0691dc658e01762a1a0d474/data,, 
> ...) failed: " at line 530 in file ..\src\condor_starter.V6.1\os_proc.cpp
> 06/19 14:24:05 ShutdownFast all jobs.
> ==============================================================
> 
> It was my understanding that, in the absence of a shared filesystem, 
> condor simply copies whatever "Executable =" points to, and transfers it 
> to \execute\dir_pid\condor_exec.exe whatever the extension is, then 
> executes that, oblivious as to the original path and filename. is this 
> not correct?
> 
> I guess I am looking for the code that generates the odd file name, and 
> work around that. One way I've found is to pre-stage the executable to 
> the gt5 resource, then condor-g submit using "transfer_executable = 
> false" so gass is never used, but still including globusrsl to enable 
> file transfer on the remote pool. This works, but kind of defeats the 
> point of having a file transfer capable meta-scheduler. Ultimately, I 
> would like to be able to use condor-g to transparantly schedule jobs on 
> multiple heterogeneous remote condor pools without having to worry about 
> file staging.
> 
> If I'm going about this the wrong way, please enlighten me!
> 
> TIA,
> 
> Rob
> 
> On 06/19/2010 09:57 AM, Burnett, Ben wrote:
> > Hi Rob:
> >
> > This is due to some embarrassingly naive file extension detection code.  It only hunts backwards from the end of the filename until it finds a '.' character, but does error out if it detects an character that is invalid in filename extension (like '<','>', ':', and in your case '/', etc/).
> >
> > As a temporary solution before either code (the one that checks for file extensions, and the one that generates that odd file name) gets patched, you can set ALLOW_SCRIPTS_TO_RUN_AS_EXECUTABLES to FALSE on the Windows machines that are experiencing this problem.  This simply tells Condor to ignore it's ability to detect a file type, and just treat it as a blain old Windows binary.
> >
> > Regards,
> > -B
> >
> > On 2010-06-18, at 3:40 PM, Rob de Graaf wrote:
> >
> >> I'm having some problems running jobs on a remote windows pool..
> >>
> >> The setup: Condor-G ->  GT5 resource w/ Condor LRMS ->  WinXP startd
> >>
> >> The remote pool doesn't have a shared filesystem, so I use globusrsl in the condor-g submit file to tell the remote pool it has to use its own file transfer mechanism. It has separate queues (globus jobmanagers) for linux and windows jobs. The linux queue works well, but in the windows queue, jobs fail to execute at the startd:
> >>
> >> ===============================================================
> >> 06/18 22:22:08 Starting a VANILLA universe job with ID: 37.0
> >> 06/18 22:22:09 Tracking process family by login "condor-reuse-slot1"
> >> 06/18 22:22:09 IWD: C:\Progra~1\Condor\execute\dir_428
> >> 06/18 22:22:09 Output file: C:\Progra~1\Condor\execute\dir_428\_condor_stdout
> >> 06/18 22:22:10 Error file: C:\Progra~1\Condor\execute\dir_428\_condor_stderr
> >> 06/18 22:22:10 Renice expr "10" evaluated to 10
> >> 06/18 22:22:10 About to exec C:\Progra~1\Condor\execute\dir_428\condor_exec.gass_cache/local/md5/32/4a5afa0e96a19fa2244e8dd70116ce/md5/f4/91c2ba79ec192a3e127340f8999f71/data
> >> 06/18 22:22:10 GetExecutableAndArgumentsByExtention: failed to find extension *.gass_cache/local/md5/32/4a5afa0e96a19fa2244e8dd70116ce/md5/f4/91c2ba79ec192a3e127340f8999f71/data in the registry (last-error =
> >> 2).
> >> 06/18 22:22:10 Create_Process(): Failed to find an executable for extension *.gass_cache/local/md5/32/4a5afa0e96a19fa2244e8dd70116ce/md5/f4/91c2ba79ec192a3e127340f8999f71/data
> >> 06/18 22:22:10 ERROR: C:\Progra~1\Condor\execute\dir_428\condor_exec.gass_cache\local\md5\32\4a5afa0e96a19fa2244e8dd70116ce\md5\f4\91c2ba79ec192a3e127340f8999f71\data.exe is not a valid Windows executable
> >> 06/18 22:22:10 ERROR "Create_Process(C:\Progra~1\Condor\execute\dir_428\condor_exec.gass_cache/local/md5/32/4a5afa0e96a19fa2244e8dd70116ce/md5/f4/91c2ba79ec192a3e127340f8999f71/data,, ...) failed: " at line
> >> 530 in file ..\src\condor_starter.V6.1\os_proc.cpp
> >> 06/18 22:22:10 ShutdownFast all jobs.
> >> ===============================================================
> >>
> >> The gass_cache part should not be there.. if I understand correctly, condor-g uses gass internally to transfer files to the gt5 resource, but once they are there and a condor_submit is generated, condor's own file transfer mechanism kicks in and the startd should never see the cache url? I've checked to make sure the executable is transferred to Condor\execute\dir_pid\condor_exec.exe on the WinXP startd, and it is.
> >>
> >> How can I make this work?
> >>
> >> TIA, Rob
> >> _______________________________________________
> >> Condor-users mailing list
> >> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> >> subject: Unsubscribe
> >> You can also unsubscribe by visiting
> >> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> >>
> >> The archives can be found at:
> >> https://lists.cs.wisc.edu/archive/condor-users/
> >
> > _______________________________________________
> > Condor-users mailing list
> > To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> > subject: Unsubscribe
> > You can also unsubscribe by visiting
> > https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> >
> > The archives can be found at:
> > https://lists.cs.wisc.edu/archive/condor-users/
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/