[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Access Problems

On 8/24/2013 7:39 PM, Andrey Kuznetsov wrote:

What is happening is that YYY talks to XXX over internal network because of
the hosts file, and machine XXX tries to authenticate machine YYY.
It first does a forward name resolution of YYY.ucsc.edu which turns out to
be because of the /etc/hosts file.
Then it does a reverse DNS lookup on YYY.ucsc.edu and returns an external
IP address of that machine.
When comparing the 2 addresses, they obviously don't match because one IP
is external, and the other is internal. This causes the following errors in
the XXX's SchedLog log file when a condor_q -g command was used on YYY:
PERMISSION DENIED to unauthenticated@unmapped from host 128.114.###.YYY for
command 1111 (QMGMT_READ_CMD), access level READ: reason: READ
authorization policy contains no matching ALLOW entry for this request;

How do you know this is what is happening? In other words, how did you observe that HTCondor forward/reverse lookups are coming from different places (/etc/hosts vs DNS) and that is the issue -- or is it just a guess?

If the above is what is happening, I would expect to see the following in warning in the logs:
 WARNING: forward resolution of XXX doesn't match YYY!
Is that warning present in the logs?

I would expect HTCondor to always use either /etc/hosts or DNS in a consistent order (dictated by /etc/nsswitch.conf) regardless of forwards or backwards, as iirc all HTCondor is doing is calling libc calls to do the lookups in both cases.

Some other questions:

- what version of HTCondor are you running?

- does a forward lookup of 128.114.###.YYY on machine xxx result in a hostname that matches *.ucsc.edu ?

How do I make XXX work with YYY over the internal network?
Is there an option to modify reverse DNS lookup to resolve YYY to an
internal IP address?
Is there an option to turn off reverse DNS lookup so that YYY is
authenticated as either YYY.ucsc.edu or instead of 128.114.###.YYY?

Some quick random thoughts/pointers:

- It may help to peruse

- On the machines that are connected to the private network (i.e. the machines with multiple interfaces), you could try setting something like
This tells HTCondor to use 128.114 interface for everything except for communication between machines that have the same private network name; for those machines it will use the 10.0.* interface.

- maybe add 128.114.* or similar settings to your ALLOW_READ / ALLOW_WRITE settings?

- perhaps temporarily add "D_HOSTNAME D_SECURITY" to the SCHEDD_DEBUG for more insight/output into the scheddlog

- it is a rather drastic step, but you can disable DNS lookups all-together in HTCondor via
   NO_DNS = True
this is probably not what you want as then HTCondor constructs host names using the host's IP address as a string... but could be an option of last resort, and maybe you do not care...

Maybe this will point you in the right direction and/or others can chime in...