[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] daemons not using IPv4 on unusable IPv6 network

we had an unexpected dual stack test run yesterday when our central node [collector+negotiator] started using IPv6 due to network misconfiguration.

So it looks like the shared port daemon picked up the IPv6 address and started advertising it.

- The Negotiator started talking to local daemons on IPv6 even though forward resolution failed [2].

	This warning is, generally, only informative.  However,

07/10/17 06:40:01 (pid:3203476) (D_ALWAYS) WARNING: forward resolution of
fe80::3a63:bbff:fe3f:59b4 doesn't match fe80::3a63:bbff:fe3f:59b4!

In addition, this particular warning (IPV6 address literal not matching itself) is spurious; we just fixed it for the 8.6.5 release:


07/10/17 06:40:01 (pid:3203476) (D_ALWAYS) PERMISSION DENIED to
condor_pool@xxxxxxxxx from host fe80::3a63:bbff:fe3f:59b4 for command 421
(Reschedule), access level DAEMON: reason: DAEMON authorization policy
contains no matching ALLOW entry for this request; identifiers used for this
host: fe80::3a63:bbff:fe3f:59b4, hostname size = 0, original ip address =

condor_pool@ is usually associated with using PASSWORD authentication. I'll guess here that the problem is that the link-local address doesn't resolve to 'gridka.de', and your ALLOW list contains 'condor_pool@xxxxxxxxx'. If your ALLOW list contains 'condor_pool@*', let me know, because we just found a problem with '*' being protocol-specific:


The main question for us is why did these components try using IPv6 anyway?

I'll have to look into this. Could you check your logs and verify the sinful they were trying to contact was


? Adding D_HOSTNAME do SCHEDD_DEBUG (or STARTD_DEBUG, etc, as appropriate) should give you a lot information about what HTCondor was trying to do.

After reading the code, it looks like HTCondor prefers public addresses over private addresses, and PREFER_IPV4 only changes the relative order of the protocol within those categories. There's another layer of preferences on top, where if the private IPv4 address is declared as a private network, and the source's private network name is the same as the target's, HTCondor will use the private network. See PRIVATE_NETWORK_NAME and PRIVATE_NETWORK_INTERFACE in the manual. I don't know off the top of my head if these will work without CCB; it may also be easier simply to disable IPv6 on machines where it's known not to work (usually meaning those that have only a link-local IPv6 address).

Should the default for ENABLE_IPV6 be FALSE if only link-local IPv6 address are found?

which seem to imply that IPv6 is not used at all unless IPv4 is not working.

Just to be clear, HTCondor doesn't know (or have any way to know) if a particular protocol is "working", only if an address of that protocol is present on any of the machine's interfaces. If you want HTCondor to never try any IPv6 address, set ENABLE_IPV6 = FALSE. Otherwise:

PREFER_IPV4: Metaconfiguration; the default value for the next four settings.

ADVERTISE_IPV4_FIRST: If true, HTCondor will advertise IPv4 in its address lists before IPV6, indicating that it prefers to be contacted over IPV4. Otherwise, HTCondor will advertise IPV6 first.

target daemon's protocol preference, as indicated above, and instead select which protocol to use based on its own preferences. By default,
HTCondor will prefer IPv6.

PREFER_OUTBOUND_IPV4: If true, when HTCondor will prefer IPv4 when
IGNORE_TARGET_PROTOCOL_PREFERENCE (above) is true. Also, if IGNORE_DNS_PROTOCOL_PREFERENCE (below) is true, HTCondor will try IPv4 addresses returned by DNS first.

IGNORE_DNS_PROTOCOL_PREFERENCE: If true, HTCondor will sort DNS replies
by protocol.  (DNS replies are normally sorted in order of the host's
contact preferences.)  By default, IPv6 addresses will be sorted first.

- ToddM