[HTCondor-users] Making use of idle time

[Matthew Mulqueen <mmulqueen@xxxxxxx>]

Good afternoon everyone,

I'm currently running two clusters for my lab.  Cluster #1 is in use all the time but we've found that cluster #2 is underutilized.  Due to the way equipment for each cluster was funded, there has to be some sort of separation maintained between them.  I'd like to be able to merge the two clusters into one cluster with two pools, turning cluster #1 into pool #1 and cluster #2 into pool #2.  I'd also like to be able to have jobs sent to pool #1 to be allowed to execute on pool #2 if all of pool #1's CPUs are in use.

I've been looking at flocking as a means to achieve this but am not sure it can be configured to suit my needs.  If I submit a job to pool #1 and request 8 cores but there are only 6 cores available, is there a way to configure flocking (or any other HTCondor feature) that will execute the job on the 6 available cores in pool #1 and on 2 cores in pool #2?  Also, should a job be submitted to pool #2 while the job from pool #1 is still executing on the 2 cores in pool #2, can the pool #2 job preempt the pool #1 job and take back those 2 cores.

Thanks,

Matt