Re: [HTCondor-users] Kerberos Permission Denied error

Mailing List Archives Public Access	UW Madison Computer Sciences Department Computer Systems Lab

On September 6, 2019 at 6:39 PM Zach Miller <zmiller@xxxxxxxxxxx> wrote:
> Hi Asvija,
>
> If you want to force all authenticated transactions to use krb, you can just set this one setting:
>
> SEC_DEFAULT_AUTHENTICATION_METHODS = KERBEROS
>
>
> But forcing any of the authentication settings to "NEVER" means that krb authentication cannot/will-not happen.
>
> (Also, there might be a typo in the config that you posted, but you should remove those lines anyway. I would start with just the above single setting.)
>
>
> Cheers,
> -zach
>
>
> ïOn 9/6/19, 2:04 AM, "HTCondor-users on behalf of Asvija B" <htcondor-users-bounces@xxxxxxxxxxx on behalf of asvijab@xxxxxxx> wrote:
>
> Dear all,
> I am trying to use Kerberos authentication for submitting jobs to HT-Condor. However on the client side the submission fails complaining 'AUTHENTICATE:1002:Failure performing handshake'. The schedd log tells that the permission was denied with this error:
> DaemonCore: PERMISSION DENIED for 1112 (QMGMT_WRITE_CMD) via TCP from host <10.180.141.148:15918> (access level WRITE)
> It is a simple setup to test the Kerberos integration with condor. The KDC is running on the same machine (10.180.141.148). The same machine has been configured to run as both condor submit node and worker nodes.
>
>
> I have given the most open options for security in the condor_config file. Following are the excerpts from condor_config file, client debug messages and the schedd log entries:
> condor_config file excerpt:
> SEC_DEFAULT_NEGOTIATION = OPTIONAL
> SEC_DEFAULT_AUTHENTICATION = NEVER
> SEC_CLIENT_AUTHENCTICATION = NEVER
> SEC_DEFAULT_AUTHENTICATION_METHODS = KERBEROS
> KERBEROS_MAP_FILE = $(RELEASE_DIR)/etc/condor.kmap
> SCHEDD.ALLOW_WRITE = *@*/*, 10.180.141.148
> SEC_WRITE_AUTHENTICATION = NEVER
>
>
>
> condor.kmap contents:
>
> [root@gridfs log]# cat /usr/local/nsg/condor/etc/condor.kmap
> NSGTEST.CDAC.IN = nsgtest.cdac.in
>
>
>
>
> Kerberos klist output on client side:
>
> [asvija@gridfs condor]$ klist
> Ticket cache: KEYRING:persistent:1005:1005
> Default principal:
> asvija@xxxxxxxxxxxxxxx <mailto:asvija@xxxxxxxxxxxxxxx>
>
> Valid starting Expires Service principal
> 09/06/2019 12:18:30 09/07/2019 12:18:30
> krbtgt/NSGTEST.CDAC.IN@xxxxxxxxxxxxxxx <mailto:krbtgt/NSGTEST.CDAC.IN@xxxxxxxxxxxxxxx>
>
> Debug output from condor_submit :
>
> [asvija@gridfs condor]$ _condor_TOOL_DEBUG=D_SECURITY condor_submit -debug condor-universe.job 2>&1 | tee out
>
>
> 09/06/19 12:21:05 KEYCACHE: created: 0x239a150
> 09/06/19 12:21:05 Can't open directory "/opt/condor//config" as PRIV_UNKNOWN, errno: 2 (No such file or directory)
> 09/06/19 12:21:05 Cannot open /opt/condor//config: No such file or directory
> Submitting job(s)09/06/19 12:21:05 CRED: NO MODULES REQUESTED
> 09/06/19 12:21:05 SECMAN: command 1112 QMGMT_WRITE_CMD to schedd at <10.180.141.148:9618> from TCP port 22376 (blocking).
> 09/06/19 12:21:05 IPVERIFY: Subsystem SUBMIT
> 09/06/19 12:21:05 IPVERIFY: Permission ALLOW
> 09/06/19 12:21:05 IPVERIFY: Subsystem SUBMIT
> 09/06/19 12:21:05 IPVERIFY: Permission READ
> 09/06/19 12:21:05 ipverify: READ optimized to allow anyone
> 09/06/19 12:21:05 IPVERIFY: Subsystem SUBMIT
> 09/06/19 12:21:05 IPVERIFY: Permission WRITE
> 09/06/19 12:21:05 ipverify: WRITE optimized to allow anyone
> 09/06/19 12:21:05 IPVERIFY: Subsystem SUBMIT
> 09/06/19 12:21:05 IPVERIFY: Permission NEGOTIATOR
> 09/06/19 12:21:05 ipverify: NEGOTIATOR optimized to allow anyone
> 09/06/19 12:21:05 IPVERIFY: Subsystem SUBMIT
> 09/06/19 12:21:05 IPVERIFY: Permission ADMINISTRATOR
> 09/06/19 12:21:05 ipverify: ADMINISTRATOR optimized to allow anyone
> 09/06/19 12:21:05 IPVERIFY: Subsystem SUBMIT
> 09/06/19 12:21:05 IPVERIFY: Permission OWNER
> 09/06/19 12:21:05 ipverify: OWNER optimized to allow anyone
> 09/06/19 12:21:05 IPVERIFY: Subsystem SUBMIT
> 09/06/19 12:21:05 IPVERIFY: Permission CONFIG
> 09/06/19 12:21:05 ipverify: CONFIG optimized to deny everyone
> 09/06/19 12:21:05 IPVERIFY: Subsystem SUBMIT
> 09/06/19 12:21:05 IPVERIFY: Permission DAEMON
> 09/06/19 12:21:05 ipverify: DAEMON optimized to allow anyone
> 09/06/19 12:21:05 IPVERIFY: Subsystem SUBMIT
> 09/06/19 12:21:05 IPVERIFY: Permission SOAP
> 09/06/19 12:21:05 ipverify: SOAP optimized to allow anyone
> 09/06/19 12:21:05 IPVERIFY: Subsystem SUBMIT
> 09/06/19 12:21:05 IPVERIFY: Permission DEFAULT
> 09/06/19 12:21:05 ipverify: DEFAULT optimized to allow anyone
> 09/06/19 12:21:05 IPVERIFY: Subsystem SUBMIT
> 09/06/19 12:21:05 IPVERIFY: Permission CLIENT
> 09/06/19 12:21:05 ipverify: CLIENT optimized to allow anyone
> 09/06/19 12:21:05 IPVERIFY: Subsystem SUBMIT
> 09/06/19 12:21:05 IPVERIFY: Permission ADVERTISE_STARTD
> 09/06/19 12:21:05 ipverify: ADVERTISE_STARTD optimized to allow anyone
> 09/06/19 12:21:05 IPVERIFY: Subsystem SUBMIT
> 09/06/19 12:21:05 IPVERIFY: Permission ADVERTISE_SCHEDD
> 09/06/19 12:21:05 ipverify: ADVERTISE_SCHEDD optimized to allow anyone
> 09/06/19 12:21:05 IPVERIFY: Subsystem SUBMIT
> 09/06/19 12:21:05 IPVERIFY: Permission ADVERTISE_MASTER
> 09/06/19 12:21:05 ipverify: ADVERTISE_MASTER optimized to allow anyone
> 09/06/19 12:21:05 AUTHENTICATE: setting timeout for <10.180.141.148:9618?addrs=10.180.141.148-9618&noUDP&sock=83499_42eb_4> to 20.
> 09/06/19 12:21:05 HANDSHAKE: in handshake(my_methods = 'KERBEROS')
> 09/06/19 12:21:05 HANDSHAKE: handshake() - i am the client
> 09/06/19 12:21:05 HANDSHAKE: sending (methods == 64) to server
> 09/06/19 12:21:05 HANDSHAKE: server replied (method = 64)
> 09/06/19 12:21:05 KERBEROS: krb5_unparse_name:
> condor@xxxxxxxxxxxxxxx <mailto:condor@xxxxxxxxxxxxxxx>
> 09/06/19 12:21:05 KERBEROS: param server princ: condor
> 09/06/19 12:21:05 KERBEROS: no user yet determined, will grab up to slash
> 09/06/19 12:21:05 KERBEROS: picked user: condor
> 09/06/19 12:21:05 Client is
> condor@xxxxxxxxxxxxxxx <mailto:condor@xxxxxxxxxxxxxxx>
> 09/06/19 12:21:05 KERBEROS: Server principal is
> condor@xxxxxxxxxxxxxxx <mailto:condor@xxxxxxxxxxxxxxx>
> 09/06/19 12:21:05 Acquiring credential for user
> 09/06/19 12:21:05 Successfully located credential cache
> 09/06/19 12:21:05 condor_write(): Socket closed when trying to write 13 bytes to schedd at <10.180.141.148:9618>, fd is 4
> 09/06/19 12:21:05 Buf::write(): condor_write() failed
> 09/06/19 12:21:05 AUTHENTICATE: method 64 (KERBEROS) failed.
> 09/06/19 12:21:05 HANDSHAKE: in handshake(my_methods = '')
> 09/06/19 12:21:05 HANDSHAKE: handshake() - i am the client
> 09/06/19 12:21:05 HANDSHAKE: sending (methods == 0) to server
> 09/06/19 12:21:05 condor_write(): Socket closed when trying to write 13 bytes to schedd at <10.180.141.148:9618>, fd is 4
> 09/06/19 12:21:05 Buf::write(): condor_write() failed
> 09/06/19 12:21:05 AUTHENTICATE: handshake failed!
> 09/06/19 12:21:05 Authentication was a FAILURE.
>
> ERROR: Failed to connect to local queue manager
> AUTHENTICATE:1002:Failure performing handshake
> AUTHENTICATE:1004:Failed to authenticate using KERBEROS
>
>
>
> Schedd log:
> 09/06/19 12:26:22 (pid:83694) ******************************************************
> 09/06/19 12:26:22 (pid:83694) ** condor_schedd (CONDOR_SCHEDD) STARTING UP
> 09/06/19 12:26:22 (pid:83694) ** /usr/local/nsg/condor/sbin/condor_schedd
> 09/06/19 12:26:22 (pid:83694) ** SubsystemInfo: name=SCHEDD type=SCHEDD(5) class=DAEMON(1)
> 09/06/19 12:26:22 (pid:83694) ** Configuration: subsystem:SCHEDD local:<NONE> class:DAEMON
> 09/06/19 12:26:22 (pid:83694) ** $CondorVersion: 8.8.4 Jul 09 2019 BuildID: 474941 $
> 09/06/19 12:26:22 (pid:83694) ** $CondorPlatform: x86_64_RedHat7 $
> 09/06/19 12:26:22 (pid:83694) ** PID = 83694
> 09/06/19 12:26:22 (pid:83694) ** Log last touched 9/6 12:26:13
> 09/06/19 12:26:22 (pid:83694) ******************************************************
> 09/06/19 12:26:22 (pid:83694) Using config source: /usr/local/nsg/condor/etc/condor_config
> 09/06/19 12:26:22 (pid:83694) Using local config sources:
> 09/06/19 12:26:22 (pid:83694) /opt/condor//condor_config.local
> 09/06/19 12:26:22 (pid:83694) config Macros = 99, Sorted = 99, StringBytes = 3606, TablesBytes = 3612
> 09/06/19 12:26:22 (pid:83694) CLASSAD_CACHING is ENABLED
> 09/06/19 12:26:22 (pid:83694) Daemon Log is logging: D_ALWAYS D_ERROR
> 09/06/19 12:26:22 (pid:83694) SharedPortEndpoint: waiting for connections to named socket 83647_c6f0_4
> 09/06/19 12:26:22 (pid:83694) DaemonCore: command socket at <10.180.141.148:9618?addrs=10.180.141.148-9618&noUDP&sock=83647_c6f0_4>
> 09/06/19 12:26:22 (pid:83694) DaemonCore: private command socket at <10.180.141.148:9618?addrs=10.180.141.148-9618&noUDP&sock=83647_c6f0_4>
> 09/06/19 12:26:22 (pid:83694) History file rotation is enabled.
> 09/06/19 12:26:22 (pid:83694) Maximum history file size is: 20971520 bytes
> 09/06/19 12:26:22 (pid:83694) Number of rotated history files is: 2
> 09/06/19 12:26:22 (pid:83694) Reloading job factories
> 09/06/19 12:26:22 (pid:83694) Loaded 0 job factories, 0 were paused, 0 failed to load
> 09/06/19 12:26:28 (pid:83694) TransferQueueManager stats: active up=0/100 down=0/100; waiting up=0 down=0; wait time up=0s down=0s
> 09/06/19 12:26:28 (pid:83694) TransferQueueManager upload 1m I/O load: 0 bytes/s 0.000 disk load 0.000 net load
> 09/06/19 12:26:28 (pid:83694) TransferQueueManager download 1m I/O load: 0 bytes/s 0.000 disk load 0.000 net load
> 09/06/19 12:27:01 (pid:83694) DaemonCore: PERMISSION DENIED for 1112 (QMGMT_WRITE_CMD) via TCP from host <10.180.141.148:26321> (access level WRITE)
>
>
> Thanks and regards,
> Asvija
>
>
> ------------------------------------------------------------------------------------------------------------
>
> [ C-DAC is on Social-Media too. Kindly follow us at:
> Facebook: https://www.facebook.com/CDACINDIA & Twitter: @cdacindia ]
>
> This e-mail is for the sole use of the intended recipient(s) and may
> contain confidential and privileged information. If you are not the
> intended recipient, please contact the sender by reply e-mail and destroy
> all copies and the original message. Any unauthorized review, use,
> disclosure, dissemination, forwarding, printing or copying of this email
> is strictly prohibited and appropriate legal action will be taken.
> ------------------------------------------------------------------------------------------------------------
>
>
>
>
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/
>

Mailing List Archives

Public Access

Re: [HTCondor-users] Kerberos Permission Denied error