Re: [HTCondor-users] Setting up HAD-enabled central Managers using IDTokens

[Fabrice Bouye <FabriceB@xxxxxxx>]

Thanks for your reply,
Yes, there were configuration bits that were left over from our previous age old host based configuration from very old recommendations.
With your additional information, and going back to the manual I could achieve a working test pool with the following configuration on both CMs:

TRUST_DOMAIN=A

DAEMON_LIST     = MASTER, COLLECTOR, NEGOTIATOR, HAD, REPLICATION

CENTRAL_MANAGER1 = A
CENTRAL_MANAGER2 = B

COLLECTOR_HOST  = $(CENTRAL_MANAGER1),$(CENTRAL_MANAGER2)

HAD_USE_SHARED_PORT = TRUE
HAD_LIST = \
$(CENTRAL_MANAGER1):$(SHARED_PORT_PORT), \
$(CENTRAL_MANAGER2):$(SHARED_PORT_PORT)

REPLICATION_USE_SHARED_PORT = TRUE
REPLICATION_LIST = \
$(CENTRAL_MANAGER1):$(SHARED_PORT_PORT), \
$(CENTRAL_MANAGER2):$(SHARED_PORT_PORT)

HAD_USE_PRIMARY = TRUE
HAD_USE_REPLICATION = TRUE

I regenerated IDTokens on both CMs afterwards.
Master, HAD and replication logs seemed fine after HTCondor restart.
I could also test that I could connect an AP and an execute machine and that flock listing and jobs handling worked good, including when A was taken offline.
Forgot to mention that all of that was under Linux.

I am now trying to setup a Windows AP/execute node that can connect to this test flock and I will probably come back in a new topic on this  matter later on.

Cheers

--------------------------------------------------------------------------------------------

Fabrice Bouyé
IT Specialist (Scientific Computing) - Fisheries, Aquaculture and Marine Ecosystems Division
Spécialiste des technologies de l'information (informatique scientifique) - Division pêche, aquaculture et écosystèmes marin
Pacific Community | Communauté du Pacifique
CPS - B.P. D5 | 98848 Noumea, New Caledonia | Nouméa, Nouvelle-Calédonie
Tel: (687) 26 20 00 | Ext: 31411 | Mob: (687) 77 91 25 | Fax: (687) 26 38 18
E: fabriceb@xxxxxxx | Website | Twitter | LinkedIn | Facebook | YouTube | Instagram

--------------------------------------------------------------------------------------------
As part of our emissions reduction strategy, please only print this email if necessary
Dans le cadre de notre stratégie de réduction des émissions, merci d'imprimer cet e-mail uniquement si nécessaire

-----Original Message-----
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Todd L Miller via HTCondor-users
Sent: Saturday, January 13, 2024 7:21 AM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Cc: Todd L Miller <tlmiller@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] Setting up HAD-enabled central Managers using IDTokens

> I am not quite sure where to go from here.

        If I'm reading that log fragment correctly, this is the master attempting to advertise to collector B.  The CollectorLog on B should have more information about why IDTOKENS failed.  My guess is is that B's ALLOW_WRITE is set to condor@$(TRUST_DOMAIN), given what you said about how you installed things.  (You can check with `condor_config_val -raw
ALLOW_WRITE`.)

        When installed with get_htcondor, TRUST_DOMAIN defaults to $(CONDOR_HOST), which you've unset.  (You can verify this with `condor_config_val -v TRUST_DOMAIN`.)  You should set TRUST_DOMAIN to the same string on each machine in your (new) pool; it'll probably work best if the string doesn't have any spaces.

        The value of TRUST_DOMAIN at the time the token was issued is embedded in the token, and that embedded value must match the configured value*.  I have no idea what the tokens on your system(s) have embedded in them now (it should be in the output of `condor_token_list`), but you may need to re-issue your pool's tokens.**  (If the tokens in your pool were issued before you added your 99-spc-cm.config, they might have a sane value that you can set TRUST_DOMAIN to and avoid re-issuing tokens.)

        Once you've done that, since A and B have the same signing key (derived from the password you installed with), they should accept each other's token, granting them the identity 'condor@$(TRUST_DOMAIN)'.  So far, so good (hopefully).  This should also allow HAD to work -- IIRC, HAD requires ADMINISTRATOR privileges, but 'condor@$(TRUST_DOMAIN)' should already be in ALLOW_ADMINISTRATOR.

-- ToddM

*: You can configure HTCondor to accept tokens from any trust domain you
    choose, but get_htcondor configures things to accept only tokens from
    $(TRUST_DOMAIN).
**: Using condor_token_create as root.
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/