Re: [Condor-users] Quill performance

[Steven Timm <timm@xxxxxxxx>]

On Mon, 13 Jul 2009, Mag Gam wrote:

> We are in the process of buying 500+ servers for our university's
> Physics lab. We want to keep track of all of our usage, therefore we
> would like to use Quill. How would Quill scale for this type of
> workload?

Some words of advice---
1) If you dig through past condor weeks, 2007, 2008, you will
find some examples of system config of people who have made it work.
2) My quill server currently is a dual quad-core machine with 500GB
of disk in a raid 10 configuration and it is working fine. (I am
sharing that quill server between 3 condor pools which have about 900
nodes, 5000 batch slots in total.
3) It is still necessary to do a weekly VACUUM FULL on the postgres
database.  Even if you have the autovacuuming turned on as specified in
the condor user guide, 
cruft will eventually build up and condor_history will get very slow
(taking hours to return sometime).
4) I have only been logging my schedd's to quill, not everything else.
5) If you are doing fancy stuff with condor_q you should be aware that
some of the job status fields that you get in normal condor_q you
don't see in quill, but you can get them via condor_q -direct schedd.
6) Really the load on quill is the number of jobs through your system,
not the number of nodes.


Steve Timm



> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/

--
------------------------------------------------------------------
Steven C. Timm, Ph.D  (630) 840-8525
timm@xxxxxxxx  http://home.fnal.gov/~timm/
Fermilab Computing Division, Scientific Computing Facilities,
Grid Facilities Department, FermiGrid Services Group, Assistant Group Leader.