[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Quill Question

I'll nod in agreement at Steve Timm's comments earlier.

To add to the discussion: we experienced scaling issues at Purdue with a central quill database, (a dual xeon E5410, 16 GB RAM, 14-drive RAID-10 for postgres) and eventually got the database to withstand the number of clients, but the real pain came if a quilld got behind doing updates..

We couldn't vacuum postgres enough (so the DB grew endlessly), postgres would deadlock, and with updates falling behind, the usefulness was limited. It did get better, in some cases: UW found a couple bottlenecks that did improve some issues considerably.

It really turned into a postgres tuning exercise, and in the end I elected not to become an DBA or move to Oracle, so we've stopped spending time on it.

Part way through the project, I talked about our experiences last year at Condor Week:

best of luck!


On Feb 18, 2009, at 3:30 AM, Henning Fehrmann wrote:

Hi Dave,

I'd like to verify that you mean 1680 slots, rather than 1680 machines with multiple slots per machine. I'm considering implementing Quill on my system, but we have about 1600 slots, so I'm concerned that we might
run into the scaling issues you're seeing.

We have 1680 machines with 4 slots each. The number users and jobs
in the queue also influences the access time of the database.
The Quill server had a E5345 Xeon processor and 16G ram.

We try to go back to Quill once we settled some other issues.
It could help to change some parameter in the postgresql.conf.
Maybe, the shape of tables and the indexing was tuned in condor v7.2.0.
One might ask the developers.
I would give it a try.

Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting

The archives can be found at:

Preston Smith    <psmith@xxxxxxxxxx>
Systems Research Engineer
Rosen Center for Advanced Computing, Purdue University