Re: [HTCondor-users] Out of ideas: optional SEC_READ_AUTHENTICATION for users

[Jason Patton <jpatton@xxxxxxxxxxx>]

Hi Carsten,

What is ALLOW_READ on the collector? If it's not already, try setting ALLOW_READ = * (even if just temporarily) and see if you can use condor_status after a reconfig.

If that doesn't work, for next steps, the condor_ping tool is often useful for debugging authentication/authorization issues. For example, to READ access to the collector, which is the issue that you're having, you could run from the access point:

$ condor_ping -type collector -name $(condor_config_val COLLECTOR_HOST) -verbose READ
Destination: Â Â Â Â Â Â Â Â collector <snipped>
Remote Version: Â Â Â Â Â Â Â$CondorVersion: 23.0.0 2023-09-25 BuildID: 677699 PackageID: 23.0.0-0.677699 RC $
Local ÂVersion: Â Â Â Â Â Â Â$CondorVersion: 23.0.0 2023-09-25 BuildID: 677699 PackageID: 23.0.0-0.677699 RC $
Session ID: Â Â Â Â Â Â Â Â Âcm3000:3027912:1696859212:153126
Instruction: Â Â Â Â Â Â Â Â READ
Command: Â Â Â Â Â Â Â Â Â Â 60020
Encryption: Â Â Â Â Â Â Â Â Ânone
Integrity: Â Â Â Â Â Â Â Â Â none
Authentication: Â Â Â Â Â Â Ânone
Remote Mapping: Â Â Â Â Â Â Âunauthenticated@unmapped

Authorized: Â Â Â Â Â Â Â Â ÂTRUEÂ

You can also add the "-debug" flag and use environment variables to set the debug level, which might give you a clearer idea of where the problem is, for example to spit out detailed security debug info:

$ _CONDOR_TOOL_DEBUG=D_SECURITY:2 condor_ping -debug -type collector -name $(condor_config_val COLLECTOR_HOST) -verbose READ

These logs would also be useful for us to help further, though you may want to redact them or send them off list.

Jason

On Mon, Oct 9, 2023 at 6:47âAM Carsten Aulbert <carsten.aulbert@xxxxxxxxxx> wrote:

Hi,

this is somewhat a follow-up to

https://www-auth.cs.wisc.edu/lists/htcondor-users/2023-September/msg00008.shtml

Going with version 23lts now, I've tried to strip out even more of our
configuration, e.g. getting rid of pool password, using id tokens
between servers and so on.

Mostly, this now works, as root I can use condor_status, condor_q
-global, as a user I can submit/condor_q the local schedd, BUT a regular
user cannot use condor_status/condor_q -better/... IOW anything which
somehow requires authentication across machines.

I've tried to get around this by setting

SEC_READ_AUTHENTICATION = OPTIONAL

on the central manager (both collector and Negotiator run there), but so
far to no avail (also SEC_READ_ENCRYPTION/SEC_READ_INTEGRITY are set to
optional, see below).

There is something I am overlooking, can anyone with keen eyes tell me
what I am missing?

Cheers

Carsten

PS: At some point I also tried the additional ANONYMOUS method in
SEC_READ_AUTHENTICATION_METHODS but that also did not help.

On the central manager:

# Parameters with names that match SEC_:
SEC_C_GAHP_WORKER_THREAD_DEFAULT_SESSION_DURATION = 1800
SEC_CLAIMTOBE_INCLUDE_DOMAIN = false
SEC_CLAIMTOBE_USER =
SEC_CREDENTIAL_REFRESH_INTERVAL = -1
SEC_CREDENTIAL_SWEEP_DELAY = 3600
SEC_CREDENTIAL_SWEEP_INTERVAL = 300
SEC_DEBUG_PRINT_KEYS = false
SEC_DEFAULT_AUTHENTICATION = required
SEC_DEFAULT_AUTHENTICATION_METHODS = FS,IDTOKENS
SEC_DEFAULT_AUTHENTICATION_TIMEOUT = 20
SEC_DEFAULT_ENCRYPTION = required
SEC_DEFAULT_INTEGRITY = required
SEC_ENABLE_IMPERSONATION_TOKENS = false
SEC_ENABLE_MATCH_PASSWORD_AUTHENTICATION = true
SEC_ENABLE_REMOTE_ADMINISTRATION = true
SEC_IMPERSONATION_TOKEN_LIMITS =
SEC_INVALIDATE_SESSIONS_VIA_TCP = true
SEC_ISSUED_TOKEN_EXPIRATION =
SEC_PASSWORD_DIRECTORY = /etc/condor/passwords.d
SEC_PASSWORD_DOMAIN =
SEC_PASSWORD_FILE = $(SEC_PASSWORD_DIRECTORY)/POOL
SEC_READ_AUTHENTICATION = OPTIONAL
SEC_READ_ENCRYPTION = OPTIONAL
SEC_READ_INTEGRITY = OPTIONAL
SEC_SCITOKENS_ALLOW_EXTRA_SLASH = false
SEC_SCITOKENS_ALLOW_FOREIGN_TOKEN_TYPES = true
SEC_SCITOKENS_CACHE = $(RUN)/cache
SEC_SCITOKENS_FOREIGN_TOKEN_ISSUERS = https://aai-dev.egi.eu/auth/realms/egi
SEC_SESSION_DURATION_SLOP = 20
SEC_SYSTEM_KNOWN_HOSTS = /etc/condor/known_hosts
SEC_TCP_SESSION_TIMEOUT = 20
SEC_TOKEN_AP_SIGNING_KEY_NAME = AP
SEC_TOKEN_DIRECTORY =
SEC_TOKEN_FETCH_ALLOWED_SIGNING_KEYS = POOL AP
SEC_TOKEN_ISSUER_KEY = POOL
SEC_TOKEN_MAX_AGE =
SEC_TOKEN_POOL_SIGNING_KEY_FILE = $(SEC_PASSWORD_FILE)
SEC_TOKEN_REQUEST_LIMITS =
SEC_TOKEN_REVOCATION_EXPR =
SEC_TOKEN_SYSTEM_DIRECTORY = /etc/condor/tokens.d
SEC_USE_FAMILY_SESSION = true


Should not be relevant, but just in case (submit host):

# Parameters with names that match SEC_:
SEC_C_GAHP_WORKER_THREAD_DEFAULT_SESSION_DURATION = 1800
SEC_CLAIMTOBE_INCLUDE_DOMAIN = false
SEC_CLAIMTOBE_USER =
SEC_CLIENT_AUTHENTICATION = OPTIONAL
SEC_CREDENTIAL_REFRESH_INTERVAL = -1
SEC_CREDENTIAL_SWEEP_DELAY = 3600
SEC_CREDENTIAL_SWEEP_INTERVAL = 300
SEC_DEBUG_PRINT_KEYS = false
SEC_DEFAULT_AUTHENTICATION = required
SEC_DEFAULT_AUTHENTICATION_METHODS = FS,IDTOKENS
SEC_DEFAULT_AUTHENTICATION_TIMEOUT = 20
SEC_DEFAULT_ENCRYPTION = required
SEC_DEFAULT_INTEGRITY = required
SEC_ENABLE_IMPERSONATION_TOKENS = false
SEC_ENABLE_MATCH_PASSWORD_AUTHENTICATION = true
SEC_ENABLE_REMOTE_ADMINISTRATION = true
SEC_IMPERSONATION_TOKEN_LIMITS =
SEC_INVALIDATE_SESSIONS_VIA_TCP = true
SEC_ISSUED_TOKEN_EXPIRATION =
SEC_PASSWORD_DIRECTORY = /etc/condor/passwords.d
SEC_PASSWORD_DOMAIN =
SEC_PASSWORD_FILE = $(SEC_PASSWORD_DIRECTORY)/POOL
SEC_READ_AUTHENTICATION = OPTIONAL
SEC_READ_ENCRYPTION = OPTIONAL
SEC_READ_INTEGRITY = OPTIONAL
SEC_SCITOKENS_ALLOW_EXTRA_SLASH = false
SEC_SCITOKENS_ALLOW_FOREIGN_TOKEN_TYPES = true
SEC_SCITOKENS_CACHE = $(RUN)/cache
SEC_SCITOKENS_FOREIGN_TOKEN_ISSUERS = https://aai-dev.egi.eu/auth/realms/egi
SEC_SESSION_DURATION_SLOP = 20
SEC_SYSTEM_KNOWN_HOSTS = /etc/condor/known_hosts
SEC_TCP_SESSION_TIMEOUT = 20
SEC_TOKEN_AP_SIGNING_KEY_NAME = AP
SEC_TOKEN_DIRECTORY =
SEC_TOKEN_FETCH_ALLOWED_SIGNING_KEYS = POOL AP
SEC_TOKEN_ISSUER_KEY = POOL
SEC_TOKEN_MAX_AGE =
SEC_TOKEN_POOL_SIGNING_KEY_FILE = $(SEC_PASSWORD_FILE)
SEC_TOKEN_REQUEST_LIMITS =
SEC_TOKEN_REVOCATION_EXPR =
SEC_TOKEN_SYSTEM_DIRECTORY = /etc/condor/tokens.d
SEC_USE_FAMILY_SESSION = true


User at submit host
$ condor_status
Error: communication error
AUTHENTICATE:1003:Failed to authenticate with any method
AUTHENTICATE:1004:Failed to authenticate using IDTOKENS
AUTHENTICATE:1004:Failed to authenticate using FS

D_SECURITY enabled collector log
10/09/23 11:39:32 DC_AUTHENTICATE: received DC_AUTHENTICATE from
<10.20.60.221:17937>
10/09/23 11:39:32 SECMAN: new session, doing initial authentication.
10/09/23 11:39:32 AUTHENTICATE: setting timeout for (unknown) to 20.
10/09/23 11:39:32 HANDSHAKE: in handshake(my_methods = 'FS,TOKEN')
10/09/23 11:39:32 HANDSHAKE: handshake() - i am the server
10/09/23 11:39:32 HANDSHAKE: client sent (methods == 2052)
10/09/23 11:39:32 HANDSHAKE: i picked (method == 4)
10/09/23 11:39:32 HANDSHAKE: client received (method == 4)
10/09/23 11:39:32 FS: client template is /tmp/FS_XXXXXXXXX
10/09/23 11:39:32 FS: client filename is /tmp/FS_XXXxQSy6k
10/09/23 11:39:32 Will return to DC because authentication is incomplete.
10/09/23 11:39:32 AUTHENTICATE_FS: used dir /tmp/FS_XXXxQSy6k, status: 0
10/09/23 11:39:32 AUTHENTICATE: method 4 (FS) failed.
10/09/23 11:39:32 HANDSHAKE: in handshake(my_methods = 'FS,TOKEN')
10/09/23 11:39:32 HANDSHAKE: handshake() - i am the server
10/09/23 11:39:32 HANDSHAKE: client sent (methods == 2048)
10/09/23 11:39:32 HANDSHAKE: i picked (method == 2048)
10/09/23 11:39:32 HANDSHAKE: client received (method == 2048)
10/09/23 11:39:32 Will return to DC to continue authentication..
10/09/23 11:39:32 PW: Server received ERROR from client, propagating
10/09/23 11:39:32 AUTHENTICATE: auth would still block
10/09/23 11:39:32 Will return to DC to continue authentication..
10/09/23 11:39:32 Error from client.
10/09/23 11:39:32 AUTHENTICATE: method 2048 (IDTOKENS) failed.
10/09/23 11:39:32 HANDSHAKE: in handshake(my_methods = 'FS,TOKEN')
10/09/23 11:39:32 HANDSHAKE: handshake() - i am the server
10/09/23 11:39:32 HANDSHAKE: client sent (methods == 0)
10/09/23 11:39:32 HANDSHAKE: i picked (method == 0)
10/09/23 11:39:32 HANDSHAKE: client received (method == 0)
10/09/23 11:39:32 DC_AUTHENTICATE: required authentication of
10.20.60.221 failed: AUTHENTICATE:1003:Failed to authenticate with any
method|AUTHENTICATE:1004:Failed to authenticate using
IDTOKENS|AUTHENTICATE:1004:Failed to authenticate using
FS|FS:1004:Unable to lstat(/tmp/FS_XXXxQSy6k)



--
Dr. Carsten Aulbert, Max Planck Institute for Gravitational Physics,
CallinstraÃe 38, 30167 Hannover, Germany, Phone +49 511 762 17185
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/