[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Permission Denied caching

On 3/6/2019 5:56 AM, Oliver Freyermuth wrote:
> Dear HTCondor experts,
> after a short (~5 minute) DNS and partial network outage today, we've 
> observed several cases of:
> PERMISSION DENIED to condor_pool@xxxxxxxxxx from host XXX.YYY.ZZZ.XXX 
> for command 2 (UPDATE_MASTER_AD), access level ADVERTISE_MASTER: reason: 
> cached result for ADVERTISE_MASTER; see first case for the full reason
> on the Central Manager (i.e. the collector), which persisted over hours. 
> It seems the cache entries never expire?
> I can not make out an automatic expiration of such DENY entries from 
> temporary DNS failures.
> Is the only way to recover from something like this a restart of the 
> collector, or am I missing something?

Hi Oliver,

The cached ALLOW/DENY entries should purged periodically (approx every 8 hours by default), or whenever an admin does a condor_reconfig.  

The condor_config knob DNS_CACHE_REFRESH can be used to change from the eight hour default; the value is in seconds.

Since you are looking at the code, note function IPVerify:refreshDNS() which is invoked upon reconfig, and also note a timer is setup to call this method periodically based on the DNS_CACHE_REFRESH knob at