[Condor-users] file transfer problem -- 2 NICs

[Jim Wilgenbusch <jwilgenb@xxxxxxxxxxx>]

I have a rocks cluster and use condor for job submission and 
distribution.  My setup is a little unusually because the CM is not 
on the rocks head node. This is causing some problems that I hope 
someone can help me with.  Here's my setup:

Rocks head node: 2 NICs.  I'll use OUT to refer to the address 
visible to the world and IN as the address visible inside the cluster.

The head node: Masterd and Schedd.
Compute nodes: Masterd and Startd.

CM (Collector and Negotiator) is outside the Rocks cluster.
Compute nodes are all visible to the CM and to other submit nodes via 
HIGHPORT/LOWPORT range.

The rocks head node has the following config:

NETWORK_INTERFACE = OUT
BIND_ALL_INTERFACES = TRUE

With this setup I can successfully submit to any startd nodes outside 
the Rocks cluster from the rocks head node and other submit nodes 
(outside the rocks cluster) can successfully run on any of the Rock 
cluster compute (startd) nodes.

The problem occurs only when I attempt to submit jobs from the rocks 
head node with the requirement that they run on the rocks compute 
nodes.  In other words:

rocks_head -> outside_startd     works
outside_schedd -> rocks_compute  works
rocks_head -> rocks_compute      DOESN'T work

After the jobs are submitted and successfully matched to one of the 
rocks compute nodes I get the following error messages:

Shadow log on submit node:

9/7 18:13:02 (692.1) (11543): ERROR "Error from starter on 
vm2@xxxxxxxxxxxxxxxxxxxx: Failed to transfer files" at line 597 in 
file pseudo_ops.C

from user submit log file:

007 (692.002.000) 09/07 18:14:42 Shadow exception!
        Error from starter on vm1@xxxxxxxxxxxxxxxxxxxx: Failed to 
transfer files

Any suggestion?

Jim




--
---

**********************************************
Jim Wilgenbusch
jwilgenb@xxxxxxxxxxxx

School of Computational Science
Florida State University
150-E Dirac Science Center Library
Tallahassee, Florida 32306-4120

Voice: 850.645.0307
Fax: 850.644.0098
**********************************************