Re: [Condor-users] Best to submit from condor pool controler orfrom clients?

["Paul Chubb" <paul.chubb@xxxxxxxxxx>]

John,
     a comment on burden for central manager. My test bed is a PIII650, 512Mb
running two VMs and the host all W2K server in a manager/submit, two exec node
config. This started to clag with 9000+ jobs. I ran about 65K jobs in six
months. Main impact was the initial submit and the logging of the jobs in the
submit log. I would expect that disk would also be an issue if the jobs were
largish. Condor_q was expensive in this scenario with a several second wait at
100% CPU. The sweet spot on this setup was about 4 - 5K jobs per submit file.

Cheers Paul



|--------+------------------------>
|        |          John Wheez    |
|        |          <john@xxxxxxxx|
|        |          om>           |
|        |                        |
|        |          02/09/2004    |
|        |          05:05 AM      |
|        |          Please respond|
|        |          to john;      |
|        |          Please respond|
|        |          to            |
|        |          Condor-Users  |
|        |          Mail List     |
|        |                        |
|--------+------------------------>
  >--------------------------------------------------------------------------|
  |                                                                          |
  |      To:     Condor-Users Mail List <condor-users@xxxxxxxxxxx>           |
  |      cc:     (bcc: Paul Chubb/Staff/ABS)                                 |
  |      Subject:     [Condor-users] Best to submit from condor pool         |
  |       controler or from     clients?                                     |
  >--------------------------------------------------------------------------|





Hi,

I'm wondering what is the better method to handle job submissions in to
my condor pool. I'm going to be having clusters composed of many jobs
and dagman consosting of many individual jobs. Is it best to have jobs
submitted from the individual clients or submit from the pool central
manager?

Rigth now i have a WWW page so that users login and submit jobs from a
WWW page which then executes condor_submit on the pool central manager
as the user.  I could easily make it so the condor_submit is executed
from the client. I'm just wondering what is best.

The benefit i can see from having the jobs submitted from the pool
central manager is that when users go to check what jobs are on the
queue, the central manager will not have to go out and query every
client to build the list of running jobs.

Or am i running the risk of burdening my central manager if it has to
keep track of a few thousand condor_submits that it made?

_______________________________________________
Condor-users mailing list
Condor-users@xxxxxxxxxxx
http://lists.cs.wisc.edu/mailman/listinfo/condor-users






-----------------------------------------------
ABS Web Site:  www.abs.gov.au