[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Problems submitting jobs from windows to Linux and vice versa



Todd,
Thanks again for your quick response, let me paste the two jobs I am trying to submit
a) Windows to Linux: 
universe = vanilla
requirements = ((Arch == "INTEL" && OpSys == "WINNT51") ||\
	(Arch == "X86_64" && OpSys == "LINUX"))
notify_user=aalas@xxxxxxxxxxxxx
environment = path=c:\condor\jobs
should_transfer_files = YES
when_to_transfer_output = ON_EXIT
executable = c:\condor\jobs\example.bat
output = c:\condor\jobs\example.out
error = c:\condor\jobs\example.err
log = c:\condor\jobs\example.log
queue 10
b) Linux to Windows:
LINUX Machine #1
Universe   = vanilla
Requirements = ((Arch == "INTEL" && OpSys == "WINNT51") ||\
               (Arch == "X86_64" && OpSys == "LINUX"))
Executable = /9/condor/jobs/simple
Arguments  = 4 10
Notify_user=kclark@xxxxxxxxxxxxx
Should_transfer_files = YES
When_to_transfer_output = ON_EXIT
Log        = /9/condor/jobs/submit.log
Output     = /9/condor/jobs/submit.out
Error      = /9/condor/jobs/submit.error
Queue 10

LINUX Machine #2
Universe   = vanilla
Requirements = ((Arch == "INTEL" && OpSys == "WINNT51") ||\
               (Arch == "X86_64" && OpSys == "LINUX"))
#Executable = /condor/jobs/simple
Executable = simple.$$(OpSys).$$(Arch)
Argument  = 4 10
Notify_user=kclark@xxxxxxxxxxxxx
Should_transfer_files = YES
When_to_transfer_output = ON_EXIT
Log        = /9/condor/jobs/submit.log
Output     = /9/condor/jobs/submit.out
Error      = /9/condor/jobs/submit.error
Queue 10
Thanks for your input in advance.

Respectfully,
Alex Alas
Systems Administrator
Fugro EarthData Inc.
Tel. 301-948-8550 x219 Fax 301-963-2064 E-mail: aalas@xxxxxxxxxxxxx 
7320 Executive Way, Frederick, MD  21704
Website: http://www.earthdata.com


-----Original Message-----
From: condor-users-bounces@xxxxxxxxxxx [mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of Todd Tannenbaum
Sent: Thursday, May 15, 2008 12:35 PM
To: Condor-Users Mail List
Subject: Re: [Condor-users] Problems submitting jobs from windows to Linux and vice versa

Alex Alas wrote:
> We can submit jobs from both platforms, but jobs run local and are
> not submitted to the condor pool for processing.  Condor_status shows
> processors being matched and claimed but the confirmation e-mails
> show that is only running on the computer were the jobs are
> submitted.
> 
> We want to deploy condor jobs across multiple platforms, but the file
> system structure will be different.  i.e. Condor installed in
> different directories.
> 
> Is there a path variable you can add to the description file or
> config file to take into account different directory structures?
>

Not sure I understand your specific issue, so I'll just type for a bit 
and discuss some solutions.  If I stumble across your problem and answer 
it, great, if not, please ask more specifically (with an example).  :).

So it doesn't matter so much where the Condor binaries are installed. 
What matters is if there is a shared file system between the nodes in 
your pool or not.

If you DO have a shared file system, and you want Condor to use it, then 
all machines should be configured with the same FILESYSTEM_DOMAIN 
setting, e.g. in condor_config put
   FILESYSTEM_DOMAIN = earthdata.com.
You could enter a command like
   condor_status -format "%s\t" Name -format "%s\n" FileSystemDomain
to see all the machines in your pool and what they are reporting as 
their file system domain.  If you submit a job telling Condor you want a 
shared file system, the Condor will only run your job on a machine where 
the submit file system domain == the execute file system domain.  This 
may explain why your jobs only run on the submission machine.

Now, lets say that your Windows machines are all on a shared network 
filesystem, and all your Linux machines also share a _different_ network 
filesystem.  For instance, all your Win32 boxes mount network drives off 
some NT server, and all your Linux boxes mount some NFS servers.  In 
this case, you'd want to put into condor_config on your Win32 boxes 
something like:
    FILESYSTEM_DOMAIN = windows.earthdata.com
and your linux boxes have in condor_config:
    FILESYSTEM_DOMAIN = linux.earthdata.com

You can specify a FileSystemDomain in the Requirements section of your 
submit description file.

If you DON'T have a shared filesystem, you can ask Condor to move the 
files are for you.  The uninspired name for this in Condor-lingo is 
"Condor's File Transfer Mechanism".

For more info and examples, see sections 2.5.3 and 2.5.4 of the Condor 
Manual - a URL is
http://www.cs.wisc.edu/condor/manual/v7.0/2_5Submitting_Job.html#SECTION00353000000000000000

Or perhaps you want the job to have a different initial working 
directory based upon where it lands?  Or ?

_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/condor-users

The archives can be found at: 
https://lists.cs.wisc.edu/archive/condor-users/