[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Condor-users] Memory leaks in Quill ?
- Date: Fri, 31 Dec 2010 09:56:53 -0600 (CST)
- From: Steven Timm <timm@xxxxxxxx>
- Subject: Re: [Condor-users] Memory leaks in Quill ?
I am also seeing all my condor 7.4.3 condor_Quill daemons using between
1.5-2 GB of RAM apiece. I don't recall that this was happening in earlier
versions of quill.
On Fri, 31 Dec 2010, Marian Zvada wrote:
On 27.12.2010 16:12, slebodnik wrote:
This problem is still actual.
I attach a graph of actual quill memory usage.
First 130 hours, value of variable QUILL_POLLING_PERIOD was "1", and
change value to "10".
We would like to also know how much memory use quill daemon on yours submit
machine (where schedd daemon is running).
I remember there is somewhere presentation from condor-week 20?? which
provides lots of useful info regarding memory footprint vs. quill usage.
Can someone, maybe from condor folks, recall this presentation and point it
to us? Or maybe provide other recommended quill setup we might use for
further tests? Qull logs don't say much using QUILL_DEBUG=D_FULLDEBUG.
Currently, we have done some valgrind and gdb work to find out source of the
trouble. Our investigation shows it's likely typical memory leak and using
simplified code as follows helps:
FILE * fp;
fp = fdopen(file_descriptor, "w") //here is memory leak
I've talked to Lukas what he found out also on our development environment -
see attached plot. What is strange, same code on CentOS takes all the memory
and crashes virtual host on our dev env. Whereas, on debian it takes 300MB of
res mem and that's it... Maybe different glibc on CentOS vs debian, but this
is what Lukas can comment more in details if needed.
Well, we have option to compile quill and see what is going on with memory in
our test case or we can live with gdb hook Lukas patched to the code. However
we would be happy if someone from condor experts can review our report and
give a hint in here. Or, should we open condor-admin ticket track the issue
Thanks in advance!
PS: Happy New Year 2011 to all condor-users and wishing successful new lines
of code into next condor releases ;)
On Tue, 7 Dec 2010 09:41:11 +0100, Vladimir Motoska <motoska@xxxxxxxx>
We have some issues with quill daemon on the submitter node. We use
condor 7.4.4 , x86_64 rhel5 dynamic running on CentOS with
The problem is that quill accumulates memory. Here is the memory usage
The data are divided to 3 columns.
1st percentage of memory used by quill
2rd usage of memory in KB
3th usage of memory in MB
Each line represents a time stamp one minute.
Here is also our condor_config.
On all other nodes quill runs just fine. Can anybody give me some hint ?
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
You can also unsubscribe by visiting
The archives can be found at:
Steven C. Timm, Ph.D (630) 840-8525
Fermilab Computing Division, Scientific Computing Facilities,
Grid Facilities Department, FermiGrid Services Group, Group Leader.
Lead of FermiCloud project.