[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[HTCondor-users] Depth-first for non-DAG and DAG jobs?

Good morning,

I've been asked why my matching patterns look rather irregular.

To keep slots as unfragmented as possible, and instead spending most-
fragmented slots first, my policy contains
also supported by a NEGOTIATOR_PRE_JOB_RANK that prefers matching with
already fragmented resources.

For "normal" jobs that works nicely, jobs get matched against the same
node until it's resources are exhausted.

I'd like to apply the same for DAGs as well, but I'm puzzled by the old
discussion here:
that somewhat discourages to set DAGMAN_SUBMIT_DEPTH_FIRST = True as well,
as I apparently cannot guarantee that there are side effects (parents
never getting to the top of the matching queue?).

Since more than 10 years have passed, has this "much easier to implement"
(as Kent stated) approach seen any brush-up, making this setting safe?
What are the recommendations for my situation nowadays?

Job runtimes are as inhomogeneous as one could imagine, and preemption and
defrag would precisely hit the wrong jobs, this is why I'd like to avoid
them as much as possible and have a clean matching pattern instead.

My last resort would be to setup special rules ("full-node requests only",
i.e. one static "cover-all" slot) for a subset of nodes, but we all know 
that walls always happen to be in the wrong places once set up...


Steffen Grunewald, Cluster Administrator
Max Planck Institute for Gravitational Physics (Albert Einstein Institute)
Am Mühlenberg 1 * D-14476 Potsdam-Golm * Germany
Fon: +49-331-567 7274
Mail: steffen.grunewald(at)aei.mpg.de