[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Efficiently using a inhomogeneous cluster




NEGOTIATOR_POST_JOB_RANK sounds like a good fit for what you are trying to do, with one proviso. The idea is that the user ranks things in some way (e.g. by Mips) that produces a bunch of top-choice machines that all have the same rank. The negotiator's "post job rank" is then used to choose the lowest memory machines from among those choices.

The problem with that plan is that if some or all of your high memory machines also happen to have high Mips, then users A and B will happen to rank them higher than low memory machines, so the negotiator's post job rank won't help. If that is a problem for you, then I can't think of a better option than to have user C explicitly identify jobs that want lots of memory. Example:

+WantBigMemory = True

Then you could use the negotiator's pre job rank (which precedes the user's specified rank rather than following it in the sort order). Example:

NEGOTIATOR_PRE_JOB_RANK = (WantBigMemory =!= True) * -Memory

That says to steer all jobs which do not want lots of memory to machines with less memory. The user's specified rank will then be used as a secondary sort to choose from the machines that were ranked highest by the negotiator's pre job rank.

--Dan

Steffen Grunewald wrote:

I'm planning to enhance part of the Condor pool by adding memory modules
to about 1/4 of the nodes, in particular since there are applications which
need more memory per process.
The same - in the short run - could of course be achieved by reducing the
number of VMs (these are dual-core machines), but the problem basically
is the same:
How do I keep other users away from the "better" nodes?

Say I've got 100 (virtual or not, doesn't matter) machines, 20 of them with larger memory resources.
Users A and B don't care because their app's memory footprint is quite
beyond the amount available. They care about CPU power though.
User C depends on big memory. CPU power isn't too important, but memory
is crucial.

Is it possible to prefer low-memory machines in negotiation in a global
way (so I don't have to depend on the cooperation of users A and B to
include some Requirement or Rank in their submit files - they will rank
by Mips or Kflops though), unless overwritten by specific Requirements (of user C who will have a memory Requirement in his submit file)?
In prose, "hand out the weakest (concerning the attribute the user didn't
specify) machine first even if there are others with close specs on the
attributes the user *did* specify".
Which is a bit tricky since Kflops and Mips are always slightly different,
and by accident the big-memory box might be a bit faster than a low-mem
one... sounds a bit like fuzzy logic...
(If it's possible to bias the user's specification by a globally defined
additional term, making the visible Mips/Kflops value less attractive,
that would be very helpful.)

Any ideas, suggestions?

Cheers,
Steffen