[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Group issues

Hi Dan, thanks for replying!

I just realized that the explanation of this behavior is simple. The priority value assigned to a job is only relevant across jobs from the same user. It turns out that when you specify an accounting group, that overrides the user for purposes of job prioritization.
So if a job is submitted as part of a group (group_name.user) than the job priorities are not considered at all?
And if that is so, is it intentional?
In our setup we're using dagmanjobid as job priority to force dag FIFO computation, but we also have to use group quota to limit job execution because of the available licenses.

Therefore, it is expected that there should be no particular ordering of jobs that are submitted under different accounting groups, even if the jobs submitted as one of the groups has a higher numerical priority than jobs submitted as the other group.
It wouldn't be such a big limitation if it only affected jobs from different accounting groups.
But job priorities of jobs assigned to the same group are not used at all.

One workaround would be to make your execution machines rank higher priority jobs higher, across all users. Example:

Rank = TARGET.JobPrio
But this would cause preemption, right? We have preemption turned off by using long preemption time,
so it would not really solve the problem.

If preemption is not enabled, the two pass negotiation algorithm (first dishing out by user priority and second by machine rank) may not do what you want, depending on the value of the user priorities.
Having a two-step preemption process to simply sort the jobs sounds a bit too complicated. Shouldn't group accounting
handle job and user priorities exactly the same way as normal matchmaking?

I'd have to think harder to come up with a workaround in that case.
That would be splendid.