Re: [HTCondor-users] Vanilla Windows pool example config files

[Phil Gregg <phil@xxxxxxxxxxxxx>]

Hi Skip,

 

Attached text file has three basic setups and example config files for A. Pool Master, B. Submit host and C. Run host.

 

Each type of host has three files condor_config, condor_config.local and condor_config.local.credd.

 

I hope they are useful.

 

Regards

 

Phil Gregg

Birkbeck

University of London

 

 

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Skip Essma (TRD)
Sent: 30 May 2019 15:36
To: htcondor-users@xxxxxxxxxxx
Subject: [HTCondor-users] Vanilla Windows pool example config files

 

Hi All,

 

I am attempting to setup a vanilla Windows pool using one PC as central manager (and able to execute jobs) and one additional PC (for now).  I have read through the entire manual but still am not clear on config file setup and credentials.  I have been unable to get the central manager to see the pool PC.  I’ve also been unsuccessful in storing credentials on the pool PC.

 

Are example config files available for a vanilla Windows pool with two or more PCs?

 

Thank you,

Skip

 

Skip Essma
Senior Race Vehicle Engineer
Toyota Racing Development U.S.A., Inc.
1125 Competition Drive
Salisbury, NC 28146
Main: (704) 216-9300; Fax: (704) 216-9301
Cell: (336) 688-7063

**************************************************************************

The information contained in this email message and any attachments may

be privileged, confidential and protected from disclosure. Any

unauthorized use, printing, copying, disclosure, dissemination of or

reliance upon this communication by persons other than the intended

recipient may be subject to legal restriction or sanction. If you think

that you have received this email message in error, please reply to the

sender and delete this email promptly. Thank you for your cooperation.

 

A. Setting up Condor on Windows - Central Manager (only done once)
Install condor from the latest condor msi file using the parameters below:
COLLECTOR_NAME = MyCollectorName
UID_DOMAIN = my.domain.com
CONDOR_ADMIN = condor_help@xxxxxxxxxxxxx
SMTP_SERVER = smtp.my.domain.com
ALLOW_READ = *.my.domain.com
ALLOW_WRITE = $(CONDOR_HOST), $(IP_ADDRESS), *.my.domain.com
ALLOW_ADMINISTRATOR = $(IP_ADDRESS)
START = FALSE
WANT_VACATE = FALSE
WANT_SUSPEND = TRUE
DAEMON_LIST = MASTER COLLECTOR NEGOTIATOR
This should result in a Central Manager condor_config file listed below.
Create the files  "condor_config.local" and "condor_config.local.credd" in the condor installation directory (normally C:\Condor). The contents of these files for the Central Manager is shown below.
Make sure that the line in condor_config says:
LOCAL_CONFIG_FILE = $(LOCAL_DIR)\condor_config.local  $(LOCAL_DIR)\condor_config.local.credd
Otherwise the condor_config.local.credd file will just be ignored.
After installation using the condor_config.local file (do these as an administrator):
a. Uncomment the ALLOW_CONFIG line in condor_config.local
b. Restart condor with "condor_restart" (DOS command)
c. Run this DOS command as administrator "condor_store_cred -c add"
d. Enter the condor_pool@xxxxxxxxxxxxx password when prompted
e. Comment out the ALLOW_CONFIG line again
f. Restart condor again with "condor_restart"


FILE: C:\condor\condor_config for a Central Manager
######################################################################
##
##  condor_config
##
##  This is the global configuration file for condor. This is where
##  you define where the local config file is. Any settings
##  made here may potentially be overridden in the local configuration
##  file.  KEEP THAT IN MIND!  To double-check that a variable is
##  getting set from the configuration file that you expect, use
##  condor_config_val -v <variable name>
##
##  condor_config.annotated is a more detailed sample config file
##
##  Unless otherwise specified, settings that are commented out show
##  the defaults that are used if you don't define a value.  Settings
##  that are defined here MUST BE DEFINED since they have no default
##  value.
##
######################################################################

##  Where have you installed the bin, sbin and lib condor directories?   
RELEASE_DIR = C:\condor

##  Where is the local condor directory for each host?  This is where the local config file(s), logs and
##  spool/execute directories are located. this is the default for Linux and Unix systems.
#LOCAL_DIR = $(TILDE)
##  this is the default on Windows sytems
LOCAL_DIR = $(RELEASE_DIR)

##  Where is the machine-specific local config file for each host?
LOCAL_CONFIG_FILE = $(LOCAL_DIR)\condor_config.local  $(LOCAL_DIR)\condor_config.local.credd
##  If your configuration is on a shared file system, then this might be a better default
#LOCAL_CONFIG_FILE = $(RELEASE_DIR)\etc\$(HOSTNAME).local
##  If the local config file is not present, is it an error? (WARNING: This is a potential security issue.)
REQUIRE_LOCAL_CONFIG_FILE = FALSE

##  The normal way to do configuration with RPMs is to read all of the
##  files in a given directory that don't match a regex as configuration files.
##  Config files are read in lexicographic order.
LOCAL_CONFIG_DIR = $(LOCAL_DIR)\config
#LOCAL_CONFIG_DIR_EXCLUDE_REGEXP = ^((\..*)|(.*~)|(#.*)|(.*\.rpmsave)|(.*\.rpmnew))$

##  Use a host-based security policy. By default CONDOR_HOST and the local machine will be allowed
use SECURITY : HOST_BASED
##  To expand your condor pool beyond a single host, set ALLOW_WRITE to match all of the hosts
#ALLOW_WRITE = *.cs.wisc.edu
##  FLOCK_FROM defines the machines that grant access to your pool via flocking. (i.e. these machines can join your pool).
#FLOCK_FROM =
##  FLOCK_TO defines the central managers that your schedd will advertise itself to (i.e. these pools will give matches to your schedd).
#FLOCK_TO = condor.cs.wisc.edu, cm.example.edu

##--------------------------------------------------------------------
## Values set by the condor_configure script:
##--------------------------------------------------------------------

CONDOR_HOST = $(FULL_HOSTNAME)
COLLECTOR_NAME = MyCollectorName
UID_DOMAIN = my.domain.com
CONDOR_ADMIN = condor_help@xxxxxxxxxxxxx
SMTP_SERVER = smtp.my.domain.com
ALLOW_READ = *.my.domain.com
ALLOW_WRITE = $(CONDOR_HOST), $(IP_ADDRESS), *.my.domain.com
ALLOW_ADMINISTRATOR = $(IP_ADDRESS)
START = FALSE
WANT_VACATE = FALSE
WANT_SUSPEND = TRUE
DAEMON_LIST = MASTER COLLECTOR NEGOTIATOR


FILE: C:\condor\condor_config.local for a Central Manager
# Add further config below
#
# To Make this a SUBMIT HOST make sure the following DAEMONS are invoked in the config file
# DAEMON_LIST = MASTER SCHEDD
# Then restart condor ("condor_off", "net stop condor", "net start condor", "condor_on")
#
# To Make this a RUN HOST make sure the following DAEMONS are invoked in the config file
# DAEMON_LIST = MASTER STARTD KBDD
# Then restart condor ("condor_off", "net stop condor", "net start condor", "condor_on")
#
# To enable "Run As Owner" from this SUBMIT host, do the following on this host
# 1. Uncomment the ALLOW_CONFIG line only below
# 2. Restart condor services with "net stop condor" "net start condor"
# 3. Run this command as administrator "condor_store_cred -c add
# 4. Enter condor_pool@xxxxxxxxxxxxx and password when prompted
# 5. Comment out the ALLOW_CONFIG line only below
# 6. Restart condor services with "net stop condor" "net start condor"
#
CREDD_HOST  = $(CONDOR_HOST):$(CREDD_PORT)
STARTER_ALLOW_RUNAS_OWNER = True
CREDD_CACHE_LOCALLY = True
SEC_CLIENT_AUTHENTICATION_METHODS = NTSSPI, PASSWORD

SEC_CONFIG_NEGOTIATION = REQUIRED
SEC_CONFIG_AUTHENTICATION = REQUIRED
SEC_CONFIG_ENCRYPTION = REQUIRED
SEC_CONFIG_INTEGRITY = REQUIRED

# ALLOW_CONFIG = $(FULL_HOSTNAME)


FILE: C:\condor\condor_config.local.credd for a Central Manager
Read the section on Windows Secure Password Storage in the Condor Manual to fully understand how it works with the CRED Daemon etc.  

http://research.cs.wisc.edu/htcondor/manual/v8.8/MicrosoftWindows.html#x75-5760008.2.3

FILE: C:\condor\condor_config.local.credd for a Central Manager
######################################################################
##
##  condor_config.credd
##
##  This is the default local configuration file for the machine
##  running the condor_credd.  You should copy this file to the
##  appropriate location and customize it for your needs.  
##
######################################################################

## Note: The following settings will need to be present in your
## global config file:
##
##   CREDD_HOST = my-credd.cs.wisc.edu
##   STARTER_ALLOW_RUNAS_OWNER = True
##   CREDD_CACHE_LOCALLY = True
##
## You'll also need to ensure that clients are configured to use
## PASSWORD authentication on any machine that can run jobs as the
## submitting user. For example,
##
##   SEC_CLIENT_AUTHENTICATION_METHODS = NTSSPI, PASSWORD
##
## And finally, you'll need to enable CONFIG-level access for all
## machines in the pool so that the pool password can be stored:
##
##   ALLOW_CONFIG = Administrator@*
##   SEC_CONFIG_NEGOTIATION = REQUIRED
##   SEC_CONFIG_AUTHENTICATION = REQUIRED
##   SEC_CONFIG_ENCRYPTION = REQUIRED
##   SEC_CONFIG_INTEGRITY = REQUIRED
##
## See the "Executing Jobs as the Submitting User" section of the
## Condor manual for further details.

## CREDD_SETTINGS

## CREDD logging settings
## Customize these if you wish.
CREDD_LOG = $(LOG)/CreddLog
CREDD_DEBUG = D_COMMAND
MAX_CREDD_LOG = 50000000

#################################################
## CREDD Expert settings
## Everyting below is for the UBER-KNOWLEDGEABLE only!
## Do not change these unless you know what you do!
#################################################


DAEMON_LIST = $(DAEMON_LIST), CREDD
#DC_DAEMON_LIST = \
#MASTER, STARTD, SCHEDD, KBDD, COLLECTOR, NEGOTIATOR, EVENTD, \
#VIEW_SERVER, CONDOR_VIEW, VIEW_COLLECTOR, HAWKEYE, CREDD, HAD

CREDD    = $(SBIN)/condor_credd.exe

# Timeout session quickly since we normally only get contacted
# once per starter
SEC_CREDD_SESSION_TIMEOUT = 10


# Set security settings so that full security to the credd is required
CREDD.SEC_DEFAULT_AUTHENTICATION =REQUIRED
CREDD.SEC_DEFAULT_ENCRYPTION = REQUIRED
CREDD.SEC_DEFAULT_INTEGRITY = REQUIRED
CREDD.SEC_DEFAULT_NEGOTIATION = REQUIRED 

# Require PASSWORD auth for password fetching
CREDD.SEC_DAEMON_AUTHENTICATION_METHODS = PASSWORD

# Only honor password fetch requests to the trusted "condor_pool" user
CREDD.ALLOW_DAEMON = condor_pool@$(UID_DOMAIN)

# Require NTSSPI for storing credentials
CREDD.SEC_DEFAULT_AUTHENTICATION_METHODS = NTSSPI




B. Setting up Condor on Windows - Submit host
Install condor using SCCM using the (config files should be as listed below)
SCCM install:

Look in \\dcsntapp02\app_deploy\Condor_881 PSADTK

Run the Deploy-Application.exe (with an account that has admin perms on
machine and access to folder) which uses the Deploy-Application.ps1 script.

After installation with the condor_config.local file (run editor and cmd as an administrator):
a. Uncomment the ALLOW_CONFIG line in condor_config.local
b. Restart condor with "condor_restart" in command window
c. Run this DOS command as administrator "condor_store_cred -c add"
d. Enter the condor_pool@xxxxxxxxxxxxx password when prompted
e. Comment out the ALLOW_CONFIG line again
f. Restart condor with "condor_restart"

Submit hosts do not show up in the condor_status command (unless they are Execute Hosts also).


FILE: C:\condor\condor_config for a Submit Host
This file is generated by the SCCM install
######################################################################
##
##  condor_config
##
##  This is the global configuration file for condor. This is where
##  you define where the local config file is. Any settings
##  made here may potentially be overridden in the local configuration
##  file.  KEEP THAT IN MIND!  To double-check that a variable is
##  getting set from the configuration file that you expect, use
##  condor_config_val -v <variable name>
##
##  condor_config.annotated is a more detailed sample config file
##
##  Unless otherwise specified, settings that are commented out show
##  the defaults that are used if you don't define a value.  Settings
##  that are defined here MUST BE DEFINED since they have no default
##  value.
##
######################################################################

##  Where have you installed the bin, sbin and lib condor directories?   
RELEASE_DIR = C:\Condor

##  Where is the local condor directory for each host?  This is where the local config file(s), logs and
##  spool/execute directories are located. this is the default for Linux and Unix systems.
#LOCAL_DIR = $(TILDE)
##  this is the default on Windows sytems
#LOCAL_DIR = $(RELEASE_DIR)

##  Where is the machine-specific local config file for each host?
LOCAL_CONFIG_FILE = $(LOCAL_DIR)\condor_config.local
##  If your configuration is on a shared file system, then this might be a better default
#LOCAL_CONFIG_FILE = $(RELEASE_DIR)\etc\$(HOSTNAME).local
##  If the local config file is not present, is it an error? (WARNING: This is a potential security issue.)
REQUIRE_LOCAL_CONFIG_FILE = FALSE

##  The normal way to do configuration with RPMs is to read all of the
##  files in a given directory that don't match a regex as configuration files.
##  Config files are read in lexicographic order.
LOCAL_CONFIG_DIR = $(LOCAL_DIR)\config
#LOCAL_CONFIG_DIR_EXCLUDE_REGEXP = ^((\..*)|(.*~)|(#.*)|(.*\.rpmsave)|(.*\.rpmnew))$

##  Use a host-based security policy. By default CONDOR_HOST and the local machine will be allowed
use SECURITY : HOST_BASED
##  To expand your condor pool beyond a single host, set ALLOW_WRITE to match all of the hosts
#ALLOW_WRITE = *.cs.wisc.edu
##  FLOCK_FROM defines the machines that grant access to your pool via flocking. (i.e. these machines can join your pool).
#FLOCK_FROM =
##  FLOCK_TO defines the central managers that your schedd will advertise itself to (i.e. these pools will give matches to your schedd).
#FLOCK_TO = condor.cs.wisc.edu, cm.example.edu

##--------------------------------------------------------------------
## Values set by the condor_configure script:
##--------------------------------------------------------------------

CONDOR_HOST = atacama.my.domain.com
UID_DOMAIN = my.domain.com
CONDOR_ADMIN = condor_help@xxxxxxxxxxxxx
SMTP_SERVER = smtp.my.domain.com
ALLOW_READ = *.my.domain.com
ALLOW_WRITE = *.my.domain.com
ALLOW_ADMINISTRATOR = $(IP_ADDRESS)
JAVA = C:\PROGRA~1\Java\JRE18~3.0_1\bin\java.exe
use POLICY : DESKTOP
DAEMON_LIST = MASTER SCHEDD

FILE: C:\condor\condor_config.local  for a Submit Host
######################################################################
##
##  condor_config.local
##
##  This is the local configuration file for condor.
#
#   Local condor config file C:\condor\condor_config.local
#
# To Make this a SUBMIT HOST make sure the following DAEMONS are invoked in the config file
# DAEMON_LIST = MASTER SCHEDD
# Then restart condor ("condor_off", "net stop condor", "net start condor", "condor_on")
#
# To Make this a Execute Host make sure the following DAEMONS are invoked in the config file
# DAEMON_LIST = MASTER STARTD KBDD
# Then restart condor ("condor_off", "net stop condor", "net start condor", "condor_on")
#
# To enable "Run As Owner" from this SUBMIT host, do the following on this host
# 1. Uncomment the ALLOW_CONFIG line only below
# 2. Restart condor services with "net stop condor" "net start condor"
# 3. Run this command as administrator "condor_store_cred -c add
# 4. Enter condor_pool@xxxxxxxxxxxxx and password when prompted
# 5. Comment out the ALLOW_CONFIG line only below
# 6. Restart condor services with "condor_restart"
#
CREDD_HOST  = $(CONDOR_HOST):$(CREDD_PORT)
STARTER_ALLOW_RUNAS_OWNER = True
CREDD_CACHE_LOCALLY = True
SEC_CLIENT_AUTHENTICATION_METHODS = NTSSPI, PASSWORD

SEC_CONFIG_NEGOTIATION = REQUIRED
SEC_CONFIG_AUTHENTICATION = REQUIRED
SEC_CONFIG_ENCRYPTION = REQUIRED
SEC_CONFIG_INTEGRITY = REQUIRED

#ALLOW_CONFIG = $(FULL_HOSTNAME)


C. Setting up Condor on Windows - Execute Host
Install condor using SCCM using the (config files should be as listed below)
SCCM install:

Look in \\dcsntapp02\app_deploy\Condor_881 PSADTK

Run the Deploy-Application.exe (with an account that has admin perms on
machine and access to folder) which uses the Deploy-Application.ps1 script.

After installation with the condor_config.local file (run editor and cmd as an administrator):
a. Uncomment the ALLOW_CONFIG line in condor_config.local
b. Restart condor with "condor_restart" in command window
c. Run this DOS command as administrator "condor_store_cred -c add"
d. Enter the condor_pool@xxxxxxxxxxxxx password when prompted
e. Comment out the ALLOW_CONFIG line again
f. Restart condor with "condor_restart"

To see whether this worked for each machine in the pool, run the command 
condor_status -f "%s\t" Name -f "%s\n" ifThenElse(isUndefined(LocalCredd),\"UNDEF\",LocalCredd)

Any rows in the output with the UNDEF string indicate machines where secure communication is not working properly. Verify that the pool password is stored correctly on these machines. 
C:\condor>condor_status -f "%s\t" Name -f "%s\n" ifThenElse(isUndefined(LocalCredd),\"UNDEF\",LocalCredd)
slot1@xxxxxxxxxxxxxxxxxxxxx     UNDEF
slot2@xxxxxxxxxxxxxxxxxxxxx     UNDEF
slot3@xxxxxxxxxxxxxxxxxxxxx     UNDEF
slot1@xxxxxxxxxxxxxxxxxxxxx     UNDEF
slot2@xxxxxxxxxxxxxxxxxxxxx     UNDEF
slot3@xxxxxxxxxxxxxxxxxxxxx     UNDEF
slot1@xxxxxxxxxxxxxxxxxxxxx     atacama.my.domain.com:9620
slot2@xxxxxxxxxxxxxxxxxxxxx     atacama.my.domain.com:9620
slot3@xxxxxxxxxxxxxxxxxxxxx     atacama.my.domain.com:9620



FILE: C:\condor\condor_config for a Execute Host
This file is generated by the SCCM install
######################################################################
##
##  condor_config
##
##  This is the global configuration file for condor. This is where
##  you define where the local config file is. Any settings
##  made here may potentially be overridden in the local configuration
##  file.  KEEP THAT IN MIND!  To double-check that a variable is
##  getting set from the configuration file that you expect, use
##  condor_config_val -v <variable name>
##
##  condor_config.annotated is a more detailed sample config file
##
##  Unless otherwise specified, settings that are commented out show
##  the defaults that are used if you don't define a value.  Settings
##  that are defined here MUST BE DEFINED since they have no default
##  value.
##
######################################################################

##  Where have you installed the bin, sbin and lib condor directories?   
RELEASE_DIR = C:\Condor

##  Where is the local condor directory for each host?  This is where the local config file(s), logs and
##  spool/execute directories are located. this is the default for Linux and Unix systems.
#LOCAL_DIR = $(TILDE)
##  this is the default on Windows sytems
#LOCAL_DIR = $(RELEASE_DIR)

##  Where is the machine-specific local config file for each host?
LOCAL_CONFIG_FILE = $(LOCAL_DIR)\condor_config.local
##  If your configuration is on a shared file system, then this might be a better default
#LOCAL_CONFIG_FILE = $(RELEASE_DIR)\etc\$(HOSTNAME).local
##  If the local config file is not present, is it an error? (WARNING: This is a potential security issue.)
REQUIRE_LOCAL_CONFIG_FILE = FALSE

##  The normal way to do configuration with RPMs is to read all of the
##  files in a given directory that don't match a regex as configuration files.
##  Config files are read in lexicographic order.
LOCAL_CONFIG_DIR = $(LOCAL_DIR)\config
#LOCAL_CONFIG_DIR_EXCLUDE_REGEXP = ^((\..*)|(.*~)|(#.*)|(.*\.rpmsave)|(.*\.rpmnew))$

##  Use a host-based security policy. By default CONDOR_HOST and the local machine will be allowed
use SECURITY : HOST_BASED
##  To expand your condor pool beyond a single host, set ALLOW_WRITE to match all of the hosts
#ALLOW_WRITE = *.cs.wisc.edu
##  FLOCK_FROM defines the machines that grant access to your pool via flocking. (i.e. these machines can join your pool).
#FLOCK_FROM =
##  FLOCK_TO defines the central managers that your schedd will advertise itself to (i.e. these pools will give matches to your schedd).
#FLOCK_TO = condor.cs.wisc.edu, cm.example.edu

##--------------------------------------------------------------------
## Values set by the condor_configure script:
##--------------------------------------------------------------------
CONDOR_HOST = atacama.my.domain.com
UID_DOMAIN = my.domain.com
CONDOR_ADMIN = condor_help@xxxxxxxxxxxxx
SMTP_SERVER = smtp.my.domain.com
ALLOW_READ = *.my.domain.com
ALLOW_WRITE = *.my.domain.com
ALLOW_ADMINISTRATOR = $(IP_ADDRESS)
JAVA = C:\PROGRA~1\Java\JRE18~3.0_1\bin\java.exe
use POLICY : DESKTOP
DAEMON_LIST = MASTER STARTD KBDD


FILE: C:\condor\condor_config.local  for a Execute Host
######################################################################
##
##  condor_config.local
##
##  This is the local configuration file for condor.

#   Local condor config file C:\condor\condor_config.local
#
# To Make this a SUBMIT HOST make sure the following DAEMONS are invoked in the config file
# DAEMON_LIST = MASTER SCHEDD
# Then restart condor ("condor_restart")
#
# To Make this a Execute Host make sure the following DAEMONS are invoked in the config file
# DAEMON_LIST = MASTER STARTD KBDD
# Then restart condor ("condor_restart")
#
# To enable "Run As Owner" from this SUBMIT host, do the following on this host
# 1. Uncomment the ALLOW_CONFIG line only below
# 2. Restart condor with "condor_restart"
# 3. Run this DOS command as administrator "condor_store_cred -c add"
# 4. Enter the condor_pool@xxxxxxxxxxxxx password when prompted
# 5. Comment out the ALLOW_CONFIG line only below
# 6. Restart condor with "condor_restart"
#
CREDD_HOST  = $(CONDOR_HOST):$(CREDD_PORT)
STARTER_ALLOW_RUNAS_OWNER = True
CREDD_CACHE_LOCALLY = True
SEC_CLIENT_AUTHENTICATION_METHODS = NTSSPI, PASSWORD

SEC_CONFIG_NEGOTIATION = REQUIRED
SEC_CONFIG_AUTHENTICATION = REQUIRED
SEC_CONFIG_ENCRYPTION = REQUIRED
SEC_CONFIG_INTEGRITY = REQUIRED

# ALLOW_CONFIG = $(FULL_HOSTNAME)