[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Rank expression for match making using a python script

> Essentially I want to check the path of the input file (not the content) and
> ask our HDFS where the files (or chunks of them) are located.
> This will give me a list of nodes that the job should run on (therefore
> reducing network traffic). So making HTCondor behave more like YARN.

If you don't need to read the content of the files, only their path
(and I am guessing that will be include in the classAds of the job,
then yes, I think the TRANSLATE HOOK is a good place to do it, or at
least to try.

I am interested. Let me know how it goes.