[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] condor_userprio: Can't locate negotiator in local pool



On 5/10/2017 7:47 AM, Michael Di Domenico wrote:
Was there ever a resolution for this? I'm seeing the same issue with
8.6.1 on rhel7


Yes. I looked over Lee's (nomad) log files he sent me, and we discovered the culprit and a workaround for the few folks impacted.

Do you have NEGOTIATOR_HOST defined in your config file? I.e. if you do

  condor_config_val -v NEGOTIATOR_HOST

on any machine where condor_userprio is failing, does it show it being defined anywhere in your config file(s) ? If not, I am not sure what is going on and we will need to investigate. But if you do have NEGOTIATOR_HOST defined, here is the workaround to make things work again:

 1. Remove the NEGOTIATOR_HOST setting from your config file(s).

2. Make sure your config file does not reference $(NEGOTIATOR_HOST) anywhere; if you do have such entries, replace $(NEGOTIATOR_HOST) with $(CONDOR_HOST).

References to NEGOTIATOR_HOST in the default condor_config you get when you install HTCondor fresh have been removed for years [1]... back 5 years ago we removed it from the default config and from the Manual, and attempted to purge it from the source code -- but somehow it landed back in the source code, perhaps due to IPv6 support...

At some point I suggest you "start fresh" by starting with a clean/default condor_config, and then just adding in your required custom changes. The "-writeconfig" option to condor_config_val could perhaps assist in this process.

Meanwhile we will attempt to purge NEGOTIATOR_HOST from the source again.

Hope this helps and apologies for the hassle,
regards,
Todd

[1] https://htcondor-wiki.cs.wisc.edu/index.cgi/tktview?tn=2929










On Tue, Mar 28, 2017 at 11:22 AM, Lee Damon <nomad@xxxxxxxxxxxxxxxxx> wrote:
Hi John,

I sent ToddT all the non-redacted information he requested. The thing
that stuck out at me was a statement that the IP address for the
negotiator is sinful. This says it's finding it just fine but objects to
something about the address itself. I'm waiting for Todd to have a
chance to look it over.

nomad

On 3/28/17 08:09 , John M Knoeller wrote:
what does condor_status -negotiator say?

The first thing condor_userprio does is try and fetch the negotiator ad from the collector so it can check
the negotiator version and see if it should talk to the collector or the negotiator for accounting information.

When it can't get a negotiator ad from the collector, this is the error it prints out.

You could also have a look in the collector log to see if the negotiator ad is being received.   Perhaps
the collector is refusing it because of your ALLOW_WRITE setting?

-tj

-----Original Message-----
From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Lee Damon
Sent: Monday, March 27, 2017 10:51 AM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: [HTCondor-users] condor_userprio: Can't locate negotiator in local pool

We just updated our condor pool using the condor-8.6.1-1.el7.x86_64 RPM
and everything is working except condor_userprio is slightly unhappy:

~ [76] ; condor_userprio
condor_userprio: Can't locate negotiator in local pool

The command works when I give it the name of the CONDOR_HOST:

~ [77] ; condor_userprio -pool redacted
Last Priority Update:  3/27 08:41
                          Effective   Priority   Res   Total Usage  Time
Since
User Name                  Priority    Factor   In Use (wghted-hrs) Last
Usage
------------------------ ------------ --------- ------ ------------
----------
ajaech@xxxxxxxxxxxxxxxxx    137201.95   1000.00    208    198133.14
<now>
------------------------ ------------ --------- ------ ------------
----------
Number of users: 1                                 208    198133.14
0+23:59


The config files haven't changed - CONDOR_HOST and NEGOTIATOR_HOST
appear to be properly set:

~ [78] ; grep ^CONDOR_HOST /etc/condor/condor_config
CONDOR_HOST             = redacted.ee.washington.edu
~ [79] ; grep ^NEGOTIATOR_HOST /etc/condor/condor_config
NEGOTIATOR_HOST = $(CONDOR_HOST)

condor_config_val(1) confirms these settings are applied.

Any hints on how I can fix this would be appreciated.
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/



--
Todd Tannenbaum <tannenba@xxxxxxxxxxx> University of Wisconsin-Madison
Center for High Throughput Computing   Department of Computer Sciences
HTCondor Technical Lead                1210 W. Dayton St. Rm #4257
Phone: (608) 263-7132                  Madison, WI 53706-1685