[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Condor-users] Quill Question
- Date: Mon, 23 Feb 2009 10:05:38 -0500
- From: Preston Smith <psmith@xxxxxxxxxx>
- Subject: Re: [Condor-users] Quill Question
I'll nod in agreement at Steve Timm's comments earlier.
To add to the discussion: we experienced scaling issues at Purdue with
a central quill database, (a dual xeon E5410, 16 GB RAM, 14-drive
RAID-10 for postgres) and eventually got the database to withstand the
number of clients, but the real pain came if a quilld got behind doing
We couldn't vacuum postgres enough (so the DB grew endlessly),
postgres would deadlock, and with updates falling behind, the
usefulness was limited. It did get better, in some cases: UW found a
couple bottlenecks that did improve some issues considerably.
It really turned into a postgres tuning exercise, and in the end I
elected not to become an DBA or move to Oracle, so we've stopped
spending time on it.
Part way through the project, I talked about our experiences last year
at Condor Week:
best of luck!
On Feb 18, 2009, at 3:30 AM, Henning Fehrmann wrote:
I'd like to verify that you mean 1680 slots, rather than 1680
with multiple slots per machine. I'm considering implementing
my system, but we have about 1600 slots, so I'm concerned that we
run into the scaling issues you're seeing.
We have 1680 machines with 4 slots each. The number users and jobs
in the queue also influences the access time of the database.
The Quill server had a E5345 Xeon processor and 16G ram.
We try to go back to Quill once we settled some other issues.
It could help to change some parameter in the postgresql.conf.
Maybe, the shape of tables and the indexing was tuned in condor
One might ask the developers.
I would give it a try.
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx
You can also unsubscribe by visiting
The archives can be found at:
Preston Smith <psmith@xxxxxxxxxx>
Systems Research Engineer
Rosen Center for Advanced Computing, Purdue University