Re: [HTCondor-users] use comndor_submit for shell script

[Todd Tannenbaum <tannenba@xxxxxxxxxxx>]

On 1/25/2014 11:17 PM, Jiande Wang wrote:
> Hi,
>    We recently setup condor system (linux), and I am doing simple test
> on it.
>
> (1) I generated an executable file (1.exe) from a simple code, then I
> used condor script
>
> Universe = vanilla
> Executable = ./1.exe
> Error= 1.err
> Output = 1.out
> Queue
>
>    This works fine.
>
> (2) then I replace 1.exe with 1.bash in the Executable, where in 1.bash
> I simply echo one line of word, this works fine, too.
>
> (3) next, I added "1.exe" in the bash script, now the bash script looks
> like
>
> #!/bin/bash
> echo "hello"
> ./1.exe
> exit
>
>   After I submit the job, the echo line "Hello" appeared in the "1.out",
> however, in "1.err" file, it shows
> "/var/lib/condor/execute/dir_8911/condor_exec.exe: line 5: ./1.exe: No
> such file or directory"
>
> I checked directory /var/lib/condor/execute, it is empty, there is no
> "dir_8911".
>
> so is there anything wrong in what I am doing in step 3? In other words,
> is this the right way to use condor submit shell script which contains
> executable files?
>   I put all shell, executable and condor files in one directory, so the
> path shouldn't be wrong.

Are all your files being stored on a shared network filesystem (so that 
the submit node and the execute node see the same files), or are you 
expecting HTCondor to transfer your job's files between the submit and 
execute nodes?

If the former (shared file system, such as if your cluster all mounts 
the same NFS servers), you will want to ideally set FILESYSTEM_DOMAIN 
and UID_DOMAIN to be the same on all the nodes that share the same file 
system mounts and /etc/passed logins.  This tells HTCondor to default to 
not transferring files between the submit node and the execute node. 
Alternatively, you could put
  should_transfer_files = NO
in your job submit file.

My guess based on the above is that HTCondor is configured such that the 
submit machine and the execute machine are in a different 
FILESYSTEM_DOMAIN, and as a result HTCondor is trying to transfer the 
job's files for you.  If this is not what you want, see above :).  If 
this what you want, the problem is HTCondor does not know about the file 
1.exe in your third test... you will need to add the line
  transfer_input_files = 1.exe
to your job submit file.

See http://goo.gl/9rbjo6 or http://goo.gl/Y1ZpMB and/or
http://goo.gl/sl7oHJ for the relevant info in the manual.

Hope this helps
Todd


> Thanks in advance,
>
> Jiande
>
>
>
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/


--
Todd Tannenbaum <tannenba@xxxxxxxxxxx> University of Wisconsin-Madison
Center for High Throughput Computing   Department of Computer Sciences
HTCondor Technical Lead                1210 W. Dayton St. Rm #4257
Phone: (608) 263-7132                  Madison, WI 53706-1685