Re: [HTCondor-users] Documentation of DAG priorities

[Brian Candler <b.candler@xxxxxxxxx>]

On 05/10/2015 22:37, Kent WENGER wrote:
> > But until that point, I couldn't find any indication that the DAG 
> node priority was in any way related to the job priority.
>
> Well, there is this:  "Priorities are propagated to children, to 
> SUBDAGs, and to the HTCondor job itself, via the JobPrio attribute in 
> the job's ClassAd."
Ah, now I have it. Thank you :-)

> > Where this becomes important is where I've made a DAG with explicit 
node priorities - influencing
> > the order in which things are run to optimise throughput - but then 
I decide that I want one DAG
> > to take priority over another. It seems now that I have to apply 
offsets to all the DAG node priorities.
>
> Well, at this point if you want to change things once the DAGs have 
been submitted, there's not really a good way to do it.

In our case we are submitting a new DAG: it's a "rush job" which needs 
to take priority over the existing work, without pre-emption. So we just 
boost the priority of nodes in the new DAG at the time it's 
created/submitted. We don't (currently) have a need to update the 
priority of an already-running DAG.

Now, one thing I find odd is how dagman propagates priorities using the 
PARENT/CHILD relationship between nodes. Why is it expected that if one 
particular node of a DAG is run at a high priority, all its children 
must run at least at that priority too? Imagine I have a DAG which 
contains a number of subgraphs like this:

An->
Cn
Bn->

If I have a preference that all the 'A' jobs run before the 'B' jobs, 
and I set a higher priority on the 'A' jobs, then I have no way to avoid 
the high priority being inherited by the 'C' jobs. But I might not want 
ready 'C' jobs to be submitted in preference to the 'B' jobs in other 
subgraphs.  It seems pretty inflexible, but no doubt there is a good 
historical use case for it.

Also: where a DAG is included within another DAG, I think it would make 
sense to *add* the priority of the enclosing subdag node to the nodes of 
the subdag.

As a real example: suppose I have a DAG with node priorities 1,2,3,4. If 
I submit it with "condor_submit_dag -priority 100 foo.dag" then all the 
nodes end up with the same priority (100). What I wanted was 
101,102,103,104. In our case we had to change the script which writes 
out the DAG to take a priority base value.

Regards,

Brian.