[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [HTCondor-users] Rank expression for match making using a python script
- Date: Thu, 11 Aug 2016 11:19:49 -0400
- From: Jose Caballero <jcaballero.hep@xxxxxxxxx>
- Subject: Re: [HTCondor-users] Rank expression for match making using a python script
> Essentially I want to check the path of the input file (not the content) and
> ask our HDFS where the files (or chunks of them) are located.
> This will give me a list of nodes that the job should run on (therefore
> reducing network traffic). So making HTCondor behave more like YARN.
If you don't need to read the content of the files, only their path
(and I am guessing that will be include in the classAds of the job,
then yes, I think the TRANSLATE HOOK is a good place to do it, or at
least to try.
I am interested. Let me know how it goes.