Re: [HTCondor-users] How to run condor with larger ulimit at boot time?

[Brian Bockelman <bbockelm@xxxxxxxxxxx>]

Hi Brian,

I think you want to look into the MAX_FILE_DESCRIPTORS HTCondor setting.  This allows you to set the max open limits (even per-daemon) from within the condor config file.

Brian

PS - I agree, most of the stack overflow answers look incorrect.  For RHEL, the proper place would be /etc/sysconfig/condor.  To tease Dmitri and Rich, I'd point out that this is something systemd got right - daemon settings go into a config file; there isn't a semantic difference between daemons started by init versus by an admin from a shell.  :)

> On Feb 6, 2015, at 6:44 AM, Brian Candler <b.candler@xxxxxxxxx> wrote:
> 
> I have htcondor 8.2.4 running under Debian Wheezy. In /etc/security/limits.conf I have increased the number of open files permitted per process, because I have jobs which open a large number of files:
> 
> --------
> *               hard    nofile          40960
> *               soft    nofile          40960
> 
> # Note: limits for root must be set explicitly, wildcards are ignored
> root            hard    nofile          40960
> root            soft    nofile          40960
> --------
> 
> The problem is: when a machine boots up and starts condor automatically, the master daemon has smaller limits. These in turn are inherited by the worker processes (which breaks my application). Simply restarting condor is sufficient to fix the problem, but if you reboot a machine without remembering to do this, it's broken.
> 
> root@dar5:~# grep "open files" /proc/$(cat /var/run/condor/condor.pid)/limits
> Max open files            1024                 4096 files
> root@dar5:~# service condor restart
> Shutting down Condor (fast-shutdown mode)...  done.
> Starting up Condor...    done.
> root@dar5:~# grep "open files" /proc/$(cat /var/run/condor/condor.pid)/limits
> Max open files            40960                40960 files
> 
> 
> The only place I can see limits.conf referenced is from /etc/pam.d/{sshd,su,cron,login}. This suggests that I get the higher limits when I start the daemon from a login session, but not when the daemon starts at system start time directly from init.  This seems to agree:
> 
> $ grep "open files" /proc/1/limits
> Max open files            1024                 4096 files
> 
> The question is, does anybody have an idea how to fix this, so that when the service starts at machine bootup time it gets the higher limits, not the low limits inherited from init?
> 
> I have found the same question asked several time before:
> http://serverfault.com/questions/374852/how-to-set-ulimits-for-a-service-starting-at-boot
> http://serverfault.com/questions/356962/where-are-the-default-ulimit-values-set-linux-centos
> http://duntuk.com/how-raise-ulimit-open-files-and-mysql-openfileslimit
> but the answers are for RedHat, and seem to be wrong anyway (limits.conf is for PAM sessions, and the sysctl is for systemwide total open files)
> 
> Thanks,
> 
> Brian.
> 
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/