RE: [Condor-users] split processors among multiple jobs

["Ian Chesal" <ICHESAL@xxxxxxxxxx>]

> is there any way, that if i submit 2 jobs at the same time 
> (each with 30 queues), so 60 jobs total, and i have 30 
> processors, that instead of processing job 1, wait till it 
> finishes and then processes job 2, i want it to split the 30 
> processors equally and give 15 processors to job 1, and 15 
> processors to job 2 right from the start ?
> 
> any way to do this ?

The default behaviour of the schedd, given all things are equal, is to
perform FIFO execution of your clusters. So the schedd wants to run
cluster 1 than cluster 2. But, you can steer the jobs in a cluster
towards particular machines with the RANK expression in your submission
ticket. Or, absolutly require jobs from a cluster to run on a specific
set of machines using the REQUIREMENTS expression. You could also submit
as two users, one for each cluster, and let the negoitator load balance
automatically for you.

If you go with the RANK expression approach you're willing to accept
some unequal sharing between the clusters, but at the gain of
potentially better resource utilization (especially if your clusters are
not the exact same set of jobs). Assuming you have half your machines
tagged with the classad attribute MyMachineGroup = 1 and the other half
has MyMachineGroup = 2. You could do this with machine names, but it
will be a rather large RANK expression. In your first cluster's ticket
file you put:

RANK = MyMachineGroup =?= 1

and in your second cluster's ticket file you put:

RANK = MyMachineGroup =?= 2

Now cluster 1 prefers machines in the group labeled "1" and cluster 2
prefers machines in the group labeled "2". This is a less restrictive
way to steer your jobs towards preferred machines. If one cluster
finished before the other, the other cluster's idle jobs could occupy
the less preferred half of machines so you get perference without under
utilization.

To make it a hard separation, with no chance that a job from cluster 1
would run on machines preferred by jobs from cluster 2 simply change
RANK to REQUIREMENTS in the above example. Now you're saying "you can
only run on machines where MyMachineGroup is defined to be 1" (or 2 for
cluster 2). Great for absolute separation, but lousy for usage if one
cluster finished before the other because the remaining idle jobs could
never move to the empty machines.

Hope that helps.

- Ian