[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Quill performance

On Mon, 13 Jul 2009, Mag Gam wrote:

We are in the process of buying 500+ servers for our university's
Physics lab. We want to keep track of all of our usage, therefore we
would like to use Quill. How would Quill scale for this type of

Some words of advice---
1) If you dig through past condor weeks, 2007, 2008, you will
find some examples of system config of people who have made it work.
2) My quill server currently is a dual quad-core machine with 500GB
of disk in a raid 10 configuration and it is working fine. (I am
sharing that quill server between 3 condor pools which have about 900
nodes, 5000 batch slots in total.
3) It is still necessary to do a weekly VACUUM FULL on the postgres
database.  Even if you have the autovacuuming turned on as specified in
the condor user guide, cruft will eventually build up and condor_history will get very slow
(taking hours to return sometime).
4) I have only been logging my schedd's to quill, not everything else.
5) If you are doing fancy stuff with condor_q you should be aware that
some of the job status fields that you get in normal condor_q you
don't see in quill, but you can get them via condor_q -direct schedd.
6) Really the load on quill is the number of jobs through your system,
not the number of nodes.

Steve Timm

Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting

The archives can be found at:

Steven C. Timm, Ph.D  (630) 840-8525
timm@xxxxxxxx  http://home.fnal.gov/~timm/
Fermilab Computing Division, Scientific Computing Facilities,
Grid Facilities Department, FermiGrid Services Group, Assistant Group Leader.