Re: [HTCondor-users] slow submission rate

[Pek Daniel <pekdaniel@xxxxxxxxx>]

Thanks, the FSYNC trick solved the issue! :)

2013/8/1 Dan Bradley <dan@xxxxxxxxxxxx>

Are you timing just condor_submit, or are you also timing job run/completion rates?

Job submissions cause the schedd to commit a transaction to $(SPOOL)/job_queue.log.  If the disk containing that is slow, submissions will be slow.  One way to verify if this is the limiting factor is to add the following to your configuration:

CONDOR_FSYNC = FALSE

Another thing to keep in mind is that if you can batch submissions of many jobs into a single submit file, there will be fewer transactions.

--Dan

On 8/1/13 10:17 AM, Pek Daniel wrote:

Hi!

I'm experimenting with condor: I'm trying to submit a lot of dummy
jobs with condor_submit from multiple submission hosts simultaneously.
I have only a single schedd. I'm trying to stresstest this schedd.
These jobs are in the vanilla universe.

The problem is that I couldn't reach better result than 4-6
submission/sec, which seems a little low. I can't see any real
bottleneck on the machine, so I suspect that it's because of some
default value of a configuration option which throttles down the
submission requests.

Any idea how to solve this?

Thanks,
Daniel

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@cs.wisc.edu with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/