[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[HTCondor-users] How to run condor with larger ulimit at boot time?



I have htcondor 8.2.4 running under Debian Wheezy. In /etc/security/limits.conf I have increased the number of open files permitted per process, because I have jobs which open a large number of files:

--------
*               hard    nofile          40960
*               soft    nofile          40960

# Note: limits for root must be set explicitly, wildcards are ignored
root            hard    nofile          40960
root            soft    nofile          40960
--------

The problem is: when a machine boots up and starts condor automatically, the master daemon has smaller limits. These in turn are inherited by the worker processes (which breaks my application). Simply restarting condor is sufficient to fix the problem, but if you reboot a machine without remembering to do this, it's broken.

root@dar5:~# grep "open files" /proc/$(cat /var/run/condor/condor.pid)/limits
Max open files            1024                 4096 files
root@dar5:~# service condor restart
Shutting down Condor (fast-shutdown mode)...  done.
Starting up Condor...    done.
root@dar5:~# grep "open files" /proc/$(cat /var/run/condor/condor.pid)/limits
Max open files            40960                40960 files


The only place I can see limits.conf referenced is from /etc/pam.d/{sshd,su,cron,login}. This suggests that I get the higher limits when I start the daemon from a login session, but not when the daemon starts at system start time directly from init. This seems to agree:

$ grep "open files" /proc/1/limits
Max open files            1024                 4096 files

The question is, does anybody have an idea how to fix this, so that when the service starts at machine bootup time it gets the higher limits, not the low limits inherited from init?

I have found the same question asked several time before:
http://serverfault.com/questions/374852/how-to-set-ulimits-for-a-service-starting-at-boot
http://serverfault.com/questions/356962/where-are-the-default-ulimit-values-set-linux-centos
http://duntuk.com/how-raise-ulimit-open-files-and-mysql-openfileslimit
but the answers are for RedHat, and seem to be wrong anyway (limits.conf is for PAM sessions, and the sysctl is for systemwide total open files)

Thanks,

Brian.