RE: [Condor-users] To set dynamic limit on the jobs on condor

["Michael Yoder" <yoderm@xxxxxxxxxx>]

> Hi Nick and others
> 
> Can you help in the following scenario.
> 
> I have got 20 jobs. All the 20 jobs are to be "compiled" and "run"
there
> fore total no. of execution are 40.
> 
> Noe run part has to be executed after the completion of the
compilation
> part i.e. after the 20 compilation, 20 run should start.
> 
> Now the cluster is having say 5 cpus
> 
> so at the compilation time no problem as it will take 4 submission to
> compile the 20 jobs.
>
> Now here what I want is when it comes to the submission of 20 run
jobs,
> out of which say 5 has been submitted to the 5 cpu's and the next 5
jobs
> of the remaining 15 will keep on pinging/asking  the cpus to check
whether
> it is free or not to submit the next lot of 5 jobs.

I think you have some misconceptions about how Condor works.  You submit
jobs into the scheduling system (the condor_schedd in the condor lingo)
and let it schedule the jobs where and how it wants.  You shouldn't be
concerned about what cpus handle what jobs.  You may want to have a look
at the "User's Guide" section of the Condor manual: 

http://www.cs.wisc.edu/condor/manual/v6.7/2_Users_Manual.html

What you really want to do is to use DAGMan, condor's job dependency
manager.  See

http://www.cs.wisc.edu/condor/manual/v6.7/2_12DAGMan_Applications.html

You'll create 40 condor jobs, and have one DAG manage them.  You can set
it up so that the first 20 jobs (the compile jobs) run to completion
before the next 20 (the run phase) start off.  Alternately, you could
have the DAG specify the "parent-child" relationship between each
compile-run pair.

Hope this helps, 

Mike Yoder
Principal Member of Technical Staff
Ask Mike: http://docs.optena.com
Direct  : +1.408.321.9000
Fax     : +1.408.321.9030
Mobile  : +1.408.497.7597
yoderm@xxxxxxxxxx

Optena Corporation
2860 Zanker Road, Suite 201
San Jose, CA 95134
http://www.optena.com