Re: [Condor-users] Quill Question

[Preston Smith <psmith@xxxxxxxxxx>]

I'll nod in agreement at Steve Timm's comments earlier.

To add to the discussion: we experienced scaling issues at Purdue with  
a central quill database, (a dual xeon E5410, 16 GB RAM, 14-drive  
RAID-10 for postgres) and eventually got the database to withstand the  
number of clients, but the real pain came if a quilld got behind doing  
updates..

We couldn't vacuum postgres enough (so the DB grew endlessly),  
postgres would deadlock, and with updates falling behind, the  
usefulness was limited.  It did get better, in some cases: UW found a  
couple bottlenecks that did improve some issues considerably.

It really turned into a postgres tuning exercise, and in the end I  
elected not to become an DBA or move to Oracle, so we've stopped  
spending time on it.

Part way through the project, I talked about our experiences last year  
at Condor Week:
    http://www.cs.wisc.edu/condor/CondorWeek2008/condor_presentations/smith_quill.ppt

best of luck!

-Preston


On Feb 18, 2009, at 3:30 AM, Henning Fehrmann wrote:

> Hi Dave,
> > I'd like to verify that you mean 1680 slots, rather than 1680  
> > machines
> > with multiple slots per machine.  I'm considering implementing  
> > Quill on
> > my system, but we have about 1600 slots, so I'm concerned that we  
> > might
> > run into the scaling issues you're seeing.
>
> We have 1680 machines with 4 slots each. The number users and jobs
> in the queue also influences the access time of the database.
> The Quill server had a E5345 Xeon processor and 16G ram.
>
> We try to go back to Quill once we settled some other issues.
> It could help to change some parameter in the postgresql.conf.
> Maybe, the shape of tables and the indexing was tuned in condor  
> v7.2.0.
> One might ask the developers.
> I would give it a try.
>
> Cheers,
> Henning
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx  
> with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/

--
Preston Smith    <psmith@xxxxxxxxxx>
Systems Research Engineer
Rosen Center for Advanced Computing, Purdue University