[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] why does htcondor change sysctl params, and why is this done outside of /etc/sysctl.{d, conf} ?

e.g. I set net.core.rmem_max to 16MiB and condor lowers it to something
very close to 10MiB (I think you screwed up a digit, it appears to be equal
to 10*1014*1024).

	Looks like I did.  Fixed.

it's clear you're just blowing the settings out of the way even when they probably meet your needs.

The default linux kernel tuning script checks before it changes /proc/sys/net/core/rmem_max (or anything else), and will only ever increase it, at least in my testing. This has been true since the tuning script has been introduced. If that part of the script isn't working for you, please contact me off-list so we can debug it; clearly shrinking any of these parameters isn't acceptable.

(2) Clearer .conf file in /etc/sysctl.d about which kernel parameters it's
changing. You write names that you made up, not the actual kernel

I've added the name of the files in /proc that the script is changing. (The kernel parameter names are hopefully very similar if not identical, but we don't actually interact with that system at all.)

In this particular example of net.core.rmem_max, I believe the action is
not meaningful for TCP-based networks unless you also set net.ipv4.tcp_rmem
to a value that takes advantage of it. e.g.  '4096 65536 16777216'.

We're actually setting rmem_max so that we can have larger UDP buffers (so that the collector drops fewer updates on the floor).

As for why we're doing it in a script and not in /etc/sysctl.*, I can think of three reasons off the top of my head:

(a) Not all of our supported systems do /etc/sysctl.*.
(b) Changes there don't take effect immediately.  (Maybe a post-install
    script in the packaging could make the changes happen?)
(c) Installs from tarballs shouldn't scale any worse than installations
    from the distribution's packaging system.

- ToddM