[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [HTCondor-users] Documentation of DAG priorities
- Date: Tue, 06 Oct 2015 11:32:40 +0100
- From: Brian Candler <b.candler@xxxxxxxxx>
- Subject: Re: [HTCondor-users] Documentation of DAG priorities
On 05/10/2015 22:37, Kent WENGER wrote:
> But until that point, I couldn't find any indication that the DAG
node priority was in any way related to the job priority.
Well, there is this: "Priorities are propagated to children, to
SUBDAGs, and to the HTCondor job itself, via the JobPrio attribute in
the job's ClassAd."
Ah, now I have it. Thank you :-)
> > Where this becomes important is where I've made a DAG with explicit
node priorities - influencing
> > the order in which things are run to optimise throughput - but then
I decide that I want one DAG
> > to take priority over another. It seems now that I have to apply
offsets to all the DAG node priorities.
> Well, at this point if you want to change things once the DAGs have
been submitted, there's not really a good way to do it.
In our case we are submitting a new DAG: it's a "rush job" which needs
to take priority over the existing work, without pre-emption. So we just
boost the priority of nodes in the new DAG at the time it's
created/submitted. We don't (currently) have a need to update the
priority of an already-running DAG.
Now, one thing I find odd is how dagman propagates priorities using the
PARENT/CHILD relationship between nodes. Why is it expected that if one
particular node of a DAG is run at a high priority, all its children
must run at least at that priority too? Imagine I have a DAG which
contains a number of subgraphs like this:
If I have a preference that all the 'A' jobs run before the 'B' jobs,
and I set a higher priority on the 'A' jobs, then I have no way to avoid
the high priority being inherited by the 'C' jobs. But I might not want
ready 'C' jobs to be submitted in preference to the 'B' jobs in other
subgraphs. It seems pretty inflexible, but no doubt there is a good
historical use case for it.
Also: where a DAG is included within another DAG, I think it would make
sense to *add* the priority of the enclosing subdag node to the nodes of
As a real example: suppose I have a DAG with node priorities 1,2,3,4. If
I submit it with "condor_submit_dag -priority 100 foo.dag" then all the
nodes end up with the same priority (100). What I wanted was
101,102,103,104. In our case we had to change the script which writes
out the DAG to take a priority base value.