[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [HTCondor-users] using hdfs with HTCondor awaring data locality
- Date: Thu, 24 Oct 2013 17:02:35 -0400 (EDT)
- From: Tim St Clair <tstclair@xxxxxxxxxx>
- Subject: Re: [HTCondor-users] using hdfs with HTCondor awaring data locality
I've pondered many options, but the most viable are:
1.) Create a classad function which determines distance(file_block, machine) and use that in a rank _expression_. This is not pretty, because you would need to do that per block.
2.) Create a YARN based application.
From: "ricky lee" <rickylee0815@xxxxxxxxx>
Sent: Thursday, October 24, 2013 3:45:09 PM
Subject: [HTCondor-users] using hdfs with HTCondor awaring data locality
I saw a link of using HDFS with HTCondor.
After deploying HDFS through condor, I want to submit a condor job with a requirement of HDFS data locality aware. For example, a job requirement should look like
Is there an easy way to express the HDFS data locality with Condor? thx.
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
You can also unsubscribe by visiting
The archives can be found at: