[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Scheduling Policies



> I need to implement a scheduling algorithm that makes the manager
submit
> jobs for the executers only during a period of the day (e.g. after
work
> 6pm to 7am)

This part of your request is easy to fill with Condor. See this FAQ
entry:

http://www.cs.wisc.edu/condor/manual/v6.8/7_2Setting_up.html#32709

...and then adjust and adapt to suit your needs.

> and implements priority based on user, user group and application ->
> Each user group must have one priority and each user must have their
> own priority. Applications must have priority. So, scheduling must be
> done taking into account these 3 priority attributes. Those policies
> cannot be implemented in the job, that is, we cannot give much power
> to the guy that will create jobs.

This is harder to do, but not impossible.

For your last request: that policies are not mutable by the end user the
only real way I know of to lock down a Condor-based system at the policy
level is to implement your own policy engine, what people refer to as a
meta-scheduler, in front of Condor. Users submit to your policy engine
and it, in turn, creates the necessary Condor submit tickets and places
jobs in your Condor pool while at the same time enforcing some arbitrary
policy on the submit tickets and job descriptions. You can roll your own
or you can buy something like the Cycle Server from Cycle Computing
(http://www.cyclecomputing.com/).

As for scheduling based on priorities this is very nearly impossible to
do with Condor without using a meta-scheduler. You can, if you have
strict control over your submissions tickets, make Condor schedule jobs
in a priority fashion. We do this here at Altera. See my presentation
from Condor Week 2006. It details just how we submit jobs to achieve a
priority scheduling system. It's not without it's faults (mainly we have
to endure preemption for our group partitioning to work) but it's
getting better all the time (mainly because the Condor team listens to
it's users and is always improving how Condor works and supports these
weird enterprise-type deployments like ours). The presentation can be
found here:

http://www.cs.wisc.edu/condor/CondorWeek2006/presentations/chesal_altera
.ppt

That should get you started. Don't get discouraged. It can be done. And
when you see what you're saving in "LSF taxes" every year the amount of
work seems well worth while.

- Ian


Confidentiality Notice.  This message may contain information that is confidential or otherwise protected from disclosure.
If you are not the intended recipient, you are hereby notified that any use, disclosure, dissemination, distribution, 
or copying of this message, or any attachments, is strictly prohibited.  If you have received this message in error, 
please advise the sender by reply e-mail, and delete the message and any attachments.  Thank you.