Re: [Condor-users] Steering Jobs to a File on HDFS

[Dvorak DDV <dvoraky@xxxxxxxxx>]

Hi Experts,

I
 see activities of using HDFS as the underlying FS for Condor. As far as
 I understand condor is copying the file out of HDFS before a job runs, 
is this correct?

Since HDFS can be exported to workloads (jobs) 
directly via e.g. Fuse. I am just wondering is it possible that, in a 
condor cluster where compute worker nodes are also HDFS storage nodes, 
condor can send the job directly to the node where the file is located 
on HDFS?

Thanks.

On Wed, Jul 21, 2010 at 4:24 PM, Yushu Yao <yao.yushu@xxxxxxxxx> wrote:

Hi Developers,

In the 2009 what't new in Condor talk ( http://www.cs.wisc.edu/condor/CondorWeek2009/condor_presentations/tannenba_roadmap_2009.ppt ), page 32, it is said:

"Thinking about how Condor can steer jobs to the data
Via a ClassAd function used in the RANK _expression_?"

Just wondering is this done now? If yes is there documentations on how to do it?

Thanks a lot!

-Yushu
+-------------------------------------------------+
| Yushu Yao
| Ph:1-510-486-4690 
|
| Lawrence Berkeley National Lab
| Mailstop 50B-6222
| 1 Cyclotron Road
| Berkeley CA 94720-8147 - USA
+-------------------------------------------------+