Re: [Condor-users] Quill performance

[Mag Gam <magawake@xxxxxxxxx>]

Thanks for the replies.

Steve: You have 1 database which is 500GB? is that correct?  How long
does the vacuum full take? and how many days worth of data are you
keeping?




On Tue, Jul 14, 2009 at 11:25 AM, Steven Timm<timm@xxxxxxxx> wrote:
> On Mon, 13 Jul 2009, Mag Gam wrote:
>
>> We are in the process of buying 500+ servers for our university's
>> Physics lab. We want to keep track of all of our usage, therefore we
>> would like to use Quill. How would Quill scale for this type of
>> workload?
>
> Some words of advice---
> 1) If you dig through past condor weeks, 2007, 2008, you will
> find some examples of system config of people who have made it work.
> 2) My quill server currently is a dual quad-core machine with 500GB
> of disk in a raid 10 configuration and it is working fine. (I am
> sharing that quill server between 3 condor pools which have about 900
> nodes, 5000 batch slots in total.
> 3) It is still necessary to do a weekly VACUUM FULL on the postgres
> database.  Even if you have the autovacuuming turned on as specified in
> the condor user guide,
> cruft will eventually build up and condor_history will get very slow
> (taking hours to return sometime).
> 4) I have only been logging my schedd's to quill, not everything else.
> 5) If you are doing fancy stuff with condor_q you should be aware that
> some of the job status fields that you get in normal condor_q you
> don't see in quill, but you can get them via condor_q -direct schedd.
> 6) Really the load on quill is the number of jobs through your system,
> not the number of nodes.
>
>
> Steve Timm
>
>
>
>> _______________________________________________
>> Condor-users mailing list
>> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
>> subject: Unsubscribe
>> You can also unsubscribe by visiting
>> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>>
>> The archives can be found at:
>> https://lists.cs.wisc.edu/archive/condor-users/
>>
>
> --
> ------------------------------------------------------------------
> Steven C. Timm, Ph.D  (630) 840-8525
> timm@xxxxxxxx  http://home.fnal.gov/~timm/
> Fermilab Computing Division, Scientific Computing Facilities,
> Grid Facilities Department, FermiGrid Services Group, Assistant Group Leader.
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/
>