[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Jobs priority



When a Schedd is running a job it is because it was given a claim to some resource, where the job is running. The Schedd, by default, can recycle that claim as many times as it likes to run other jobs. Those other jobs need to be similar enough to the one the claim was originally for. Being in the same cluster is often similar enough. This is typically a good thing as it avoids negotiation overhead for your jobs.

There are a few ways you can stop a Schedd from recycling claims. One is a negotiation cycle can give the claimed resource to another user for their jobs, based on user priorities and negotiation policy. This effectively breaks the claim. If you're submitting cluster A, B and C to the same user, or have preemption disabled, this won't help you much. A second way is to limit the amount of time a claim can be recycled, via the CLAIM_WORKLIFE configuration option.

http://www.cs.wisc.edu/condor/manual/v7.1/3_3Configuration.html#13583

So you know, the profile of your clusters matters. For instance, if they are all very short running jobs you may in fact see them run sequentially because the A cluster may completely finish before a negotiation cycle can give resources to the B cluster. Also, when you say "immediately after" the time actually matters because if there were no negotiation cycle between A being submitted and B then you may in fact get a mixture of A and B jobs run.

Best,


matt

Daniel Tardón wrote:
Hello, thank you for your response. Im reopening this thread because ive
been absent.
I think i havent explained the situation properly.
Our problem is that we always send jobs from the same machine and the same
user.
Our problem is not prioritizing some jobs over others, what we want to do
is to get all the jobs to execute concurringly when they start arriving at
the pool.
In other words again, if we have a pool with 10 machines, a cluster A is
submitted to the pool with 100 jobs. Then immediatly after a cluster B is
submitted to the pool with 30 jobs for example. Now there are 10 jobs
running and all are from cluster A. One job finishes, and the next job to
be executed is always a job from cluster A. This is what we dont want.

We want to imitate a round-robin scheduling with the jobs of different
clusters.

We want the jobs from the cluster B to execute at the same time as jobs
from cluster A.
Not to execute cluster B only when cluster A has finished.

We would like the same thing to happen if a new cluster C entered in the
queue, so that, for example, the first job of cluster C could execute
before the last job of cluester A (obviously depending on when cluster C
was submitted).

We want the resources to be shared equaly.

I hope our doubts are clearly expressed, and that someone can help us.
Thank you very much.

Steven Timm wrote:
You can either add

Priority=nnn

to the submit file of the second job,
Could it be used in the config file of the submit host so that any job
submitted form that particular host always has the higher priority?

Cheers,
Santanu
_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/condor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/condor-users/