RE: [Condor-users] Running "batch" jobs on different platforms

[bgore@xxxxxxxxxx]

Matt, I think Condor is a little smarter. What I've noticed is that it can tell if the file is a binary and exec it appropriately. If it is a text file, you MUST have an appropriate shebang line (ie: #!/bin/sh) or some such so Condor knows how to exec it, but the name of the file or extension is not relevant on Unix (ie: you can have a file.bat that is really a shell or Perl program and Condor will run it just fine -- we do that all the time for consistent file naming that caters to the silly OSes (ie Windows) that key off the extension).

Mike, it sounds like you have a file xfer problem based on the error message "no such file or directory", which is a little puzzling, since you did force file xfer. I'd check the Shadow log on the submitter and the Start/Starter logs on the execute machine.

Also, watch out for unix/dos file formats. I've had some trouble with Windows to Unix submissions when the file is in the DOS format.

But, multiplatform execution from a single submitter works just fine, as you wish it to (just need to get those nasty details worked out).

~Brooklin

-----Original Message-----
From: condor-users-bounces@xxxxxxxxxxx [mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of Matt Hope
Sent: Monday, August 16, 2004 11:08 AM
To: Condor-Users Mail List
Subject: Re: [Condor-users] Running "batch" jobs on different platforms


Sounds like the not terribly well documented "guess
the exe type" that condor does.

if you submit a bat file on windows with suffix ".bat"
it works out that you are using a bat file and
transfers accordingly (attempting to run a renamed
copy of it so don't expect the name of the script to
be the same when it runs!), otherwise it thinks you
are using an exe and behaves differently.

on unix I guess it treats .sh the same way, but thinks
you bat file is an executable, thus sending a file
that isn't an exe renamed as if it was.

I haven't used the unix version so YMMV

I am suspect that you are submitting a bat file to a
unix box - call it a shell file instead and se what
happens...

I dislike this functionality since it seems impossible
to override...
...not to mention does it treat .cmd files in the same
way? how about .pl? etc. etc.

Suggest a sep flag "treat_exe_as_script" or
somesuch...

This is all with 6.6.x no idea if it's nicer in 6.7.x
land..

 --- Mike Frederick <Mike@xxxxxxxxxx> wrote: 
> All,
>  
> OK, I've decided that as usual I am asking the wrong questions.  Let 
> me tell you what I want to do and you guys tell me how to do
> it...
>  
> I want to be able to run pre-built "batch" jobs on
> different machines in a
> Condor pool.  By "pre-built", I mean that I will
> have a file of commands
> that perform a specific task on a machine that
> resides on that machine.  If
> I log onto the machine directly and execute the file
> as a shell procedure on
> Red Hat--or a batch file on Windows--the commands
> are executed and the task
> is completed.
>  
> In a Windows-only environment we have a set of batch
> files sitting on the
> box which we can fire-off.  The problem is that now
> we need to integrate a
> Red Hat (and eventually a Sun) box into this
> environment so that we issue a
> command and the Windows box runs its batch files,
> the Red Hat box runs its
> set of commands and the Sun box runs its set of
> commands.  It appeared to me
> that Condor (with the possible future addition of
> DAGman) could perform the
> control function of this process.
>  
> I thought I could build/debug/test each the system's independent 
> "batch" procedures, build a Condor submit file for each
> procedure which was
> configured in such a way so that each submit job
> would only run on its
> appropriate system and run the local "batch" file
> and wait for all jobs to
> complete.
>  
> Is this doable?  The Windows portion works fine; I
> have a small batch file
> defined for 2 Windows boxes in the pool; I built a
> submit file to direct
> each system to run one copy and it all works.  But
> what about Unixes?  I
> though if I built a small shell script and submitted
> it to run on the Red
> Hat box in the pool it would work.  But it doesn't,
> I get the log file you
> see below.  I'll include all the appropriate files:
>  
> ====================================================
> The shell script to be executed on Red Hat:
>  
> #!/bin/csh
> echo "Howdy!"
> echo "Here is the output from 'hostname':"
> hostname -v -i
>  
> echo ""
> echo "Output from 'ls' command:"
> ls -la
>  
> echo ""
> echo "Output from a 'ping' command:"
> ping -c 4 stargate.nuview.com
>  
> echo "That's all folks!"
>  
>
=====================================================
> The submit file for Red Hat:
>  
> universe       = vanilla
> 
> requirements   = OpSys == "LINUX"
> should_transfer_files = YES
> when_to_transfer_output = ON_EXIT
>  
> executable     = linux.bat
> output         = linux.out
> error          = linux.err
> log            = linux.log
> queue
> 
>
======================================================
>  
> The resultant log file:
>  
> 001 (012.000.000) 08/16 10:24:17 Job executing on
> host: <192.168.1.222:1620>
> ...
> 007 (012.000.000) 08/16 10:24:17 Shadow exception!
>             Error from starter on
> Mike_RH.nuview.com: Failed to execute
>
'/opt/condor-6.6.6/home/execute/dir_20253/condor_exec.exe
>                    condor_exec.exe': No such file or directory
>              0  -  Run Bytes Sent By Job
>              246  -  Run Bytes Received By Job
> ...
>  
>
======================================================
>  
> So am I just going at this all wrong?  Am I using
> Condor in a way that it
> was not intended?  Are there other software
> solutions I should consider
> other than Condor?  Any help appreciated!
>  
>  
> --
> Mike Frederick
> mike@xxxxxxxxxx <mailto:mike@xxxxxxxxxx> 
>  
> > _______________________________________________
> Condor-users mailing list
> Condor-users@xxxxxxxxxxx
>
http://lists.cs.wisc.edu/mailman/listinfo/condor-users
>  


	
	
		
___________________________________________________________ALL-NEW Yahoo! Messenger - all new features - even more fun!  http://uk.messenger.yahoo.com
_______________________________________________
Condor-users mailing list
Condor-users@xxxxxxxxxxx
http://lists.cs.wisc.edu/mailman/listinfo/condor-users