That’s a bit surprising, the *default* behavior of condor_submit is to submit to the local schedd. configuring SCHEDD_HOST is normally
something you do when you want to have it default to a non-local schedd instead.
It should be showing all of your schedds.
condor_status -schedd -autoformat Name MyAddress
that should show the names and addresses of all of your schedds.
When you run “condor_submit -name” using the Name field from the condor_status output above, that
should submit to the schedd of that name.
It does that by doing the equivalent of that condor_status query.
You can add -debug to condor_submit, and set _CONDOR_TOOL_DEBUG environment variable to see a log of the communication, like this
_CONDOR_TOOL_DEBUG=D_COMMAND _condor_submit -debug -name <submitfile>
where <submitfile> is the name of your submit file. This will log what schedd it is actually talking to. Look for QMGMT_WRITE_CMD, that is the actual submit. There will likely be a command before that where it queries the collector
to get the address of the schedd.
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx>
On Behalf Of COULTER, JAMES A CTR USAF AFMC 96 SK/CCI via HTCondor-users
Sent: Friday, January 31, 2020 1:28 PM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Cc: COULTER, JAMES A CTR USAF AFMC 96 SK/CCI <james.coulter.2.ctr@xxxxxxxxx>
Subject: Re: [HTCondor-users] Assistance Configuring Authentication
Thanks that’s very helpful. I am still having problems getting condor_submit to send jobs to the schedd running on the workstation it was submitted from. I have tried setting SCHEDD_HOST in the local config
file as well as trying condor_submit –name <schedd_name>. No errors are reported but jobs are still submitted to one schedd only.
Any suggestions on what else I can try to force condor_submit to send the job to it’s own schedd and not another workstations?
will show you the name of the local/default schedd (it’s at this top of the output)
will show you the names of all of the schedd’s in your pool
Thanks for reply. SCHEDD_HOST is undefined. The documentation says it is set with name@hostname, but for the life of me I can't figure out what the schedd daemons' names are . I ran condor_submit -verbose, but I can't find a
name attribute in the output. Can you tell me how to determine the name of the schedd?
condor_submit decides what schedd to send jobs to by looking at
the -remote option passed on the command line
the configured value of SCHEDD_HOST
looking for a local schedd (the configuration variable SCHEDD_ADDRESS_FILE tells it where to look)
If you are talking to a non-local schedd unexpectedly, you should check your configuration for SCHEDD_HOST by running
condor_config_val -verbose SCHEDD_HOST
I should warn you that CLAIMTOBE is *very* dangerous in this context. CLAIMTOBE “authentication” allows any user to impersonate any other user.
and since condor_submit can ask the schedd to run arbitrary code, CLAIMTOBE would let any user run code as any other user.
Thanks for the reply. FS_REMOTE_DIR does not seem to be recognized. Using CLAIMTOBE does allow one workstation to submit it's job to the other workstation's schedd. I can now submit jobs from both workstations.
I am curious how condor_submit determines which schedd to send a job to. Or is that maybe the negotiator on the master? I think the preferable way (at least for us) would be for condor_submit to send the job to the schedd on the
workstation the job was sent from.
I do not know how condor_submit determines which schedd to
>If the FS issue is that /tmp isn't available to the Schedd you can change
>where it writes the test file with FS_REMOTE_DIR.
>If you want to test the rest of the configuration you can try temporarily
>using the CLAIMTOBE method, which trusts the client and is insecure.
>SEC_DEFAULT_AUTHENTICATION = OPTIONAL
>SEC_DEFAULT_AUTHENTICATION_METHODS = CLAIMTOBE
>I don't have experience with PASSWORD authentication, so hopefully someone
>more knowledgeable can help you troubleshoot that.
On Wed, Jan 29, 2020 at 9:22 AM COULTER, JAMES A CTR USAF AFMC 96 SK/CCI
via HTCondor-users <htcondor-users@xxxxxxxxxxx> wrote:
> I have a requirement to configure HTCondor to submit jobs to Mac Sierra
> workstations. So far I have installed a condor master running Master,
> Collector, and Negotiator daemons on a RHEL 7 server. I have installed
> htcondor on two Mac Sierra workstations both running startd and schedd
> daemons. We created a condor user and home on our NFS file system that all
> machines can access.
> The problem I'm having is the workstations are both submitting jobs to the
> same schedd. Sometimes its to workstation A, sometimes to workstation B.
> If workstation A submits its job to B's schedd (and vice versa) I get an
> authentication error.
> I have tried several different authentication methods, but I can't get any
> to work. If I leave SEC_DEFAULT_AUTHENTICATION = OPTIONAL, I get a
> Kerberos authentication failed error.
> Right now I am trying the pool password configuration I found in the FAQ:
> This setup results in FS authentication failing when workstation A submits
> its job to schedd on workstation B.
> Workstations A and B are using the same condor_config.local file. Here
> are the contents:
> DAEMON_LIST=MASTER STARTD SCHEDD
> ALLOW_NEGOTIATOR = master.example.com
> CONDOR_IDS = 3055.8186
> CONDOR_ADMIN = root@$(FULL_HOSTNAME)
> ALL_DEBUG = D_FULLDEBUG
> # From
> SEC_PASSWORD_FILE = /etc/condor/condor_pool_password (NOTE: this file was
> created on the master and copied to both clients, owner root, mode 0600)
> SEC_DAEMON_INTEGRITY = REQUIRED
> SEC_DAEMON_AUTHENTICATION = REQUIRED
> SEC_DAEMON_AUTHENTICATION_METHODS = PASSWORD
> SEC_NEGOTIATOR_INTEGRITY = REQUIRED
> SEC_NEGOTIATOR_AUTHENTICATION = REQUIRED
> SEC_NEGOTIATOR_AUTHENTICATION_METHODS = PASSWORD
> SEC_CLIENT_AUTHENTICATION_METHODS = FS,PASSWORD
> ALLOW_DAEMON = condor_pool@*
> Here's the errors found in SchedLog after workstation A tries to submit a
> job to schedd on workstation B:
> 01/29/20 10:28:56 (pid:14019) DC_AUTHENTICATE: authentication of
> <xxx.xxx.xxx.117:58966> did not result in a valid mapped user name, which
> is required for this command (1112 QMGMT_WRITE_CMD), so aborting.
> 01/29/20 10:28:56 (pid:14019) DC_AUTHENTICATE: reason for authentication
> failure: AUTHENTICATE:1003:Failed to authenticate with any
> method|AUTHENTICATE:1004:Failed to authenticate using FS|FS:1004:Unable to
> This is what failure on a client looks like:
> coulter@albatross ~/condor>/opt/condor/bin/condor_submit -debug sleep.sub
> 01/29/20 10:28:59 Reading condor configuration from
> 01/29/20 10:28:59 Enumerating interfaces: lo0 127.0.0.1 up
> 01/29/20 10:28:59 Enumerating interfaces: lo0 ::1 up
> 01/29/20 10:28:59 Enumerating interfaces: lo0 fe80::1 up
> 01/29/20 10:28:59 Enumerating interfaces: en0 xxx.xxx.xxx.117 up
> Submitting job(s)01/29/20 10:28:59 SharedPortClient: sent connection
> request to schedd at <xxx.xxx.xxx.235:9618> for shared port id 1964_6748_6
> 01/29/20 10:28:59 SECMAN: required authentication with schedd at
> <xxx.xxx.xxx.235:9618> failed, so aborting command QMGMT_WRITE_CMD.
> ERROR: Failed to connect to local queue manager
> AUTHENTICATE:1003:Failed to authenticate with any method
> AUTHENTICATE:1004:Failed to authenticate using FS
> The way I read this is FS authentication is attempting to read a file on
> the schedd's local file system but because it isn't the submitter's local
> file system it fails. I don't see anything at all about Password
> authentication. I tried setting SEC_CLIENT_AUTHENTICATION_METHODS =
> PASSWORD but that results in AUTHENTICATE:1003:Failed to authenticate
> with any method.
> Any suggestions on what I can do? My customer has a grand total of 20 Mac
> Sierra workstations they want in the pool and we are on a dedicated network
> so security is not as high on the priority list as getting this working.
> HTCondor-users mailing list
> To unsubscribe, send a message to
> subject: Unsubscribe
> You can also unsubscribe by visiting
> The archives can be found at:
*Collin Mehring *| PE-JoSE - Software Engineer
-------------- next part ---------