[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] condor_userprio: Can't locate negotiator in local pool



Still not working here.

I ran this command and here are the results. Â

# condor_config_val -v NEGOTIATOR_HOST
Not defined: NEGOTIATOR_HOST

This is an install from within the last 3 years so shouldn't be related to the pre-purge installs.

I am getting this permission error, running as root, in the NegotiatorLog though.

ÂPERMISSION DENIED to unauthenticated@unmapped from host ---.---.---.--- for command 459 (SetPriorityFactor), access level ADMINISTRATOR: reason: cached result for ADMINISTRATOR; see first case for the full reason
05/10/17 12:33:50 DC_AUTHENTICATE: Command not authorized, done!

Thanks you for you help.



Thanks

Jon Knudson
IT Specialist
Upper Midwest Environmental Sciences Center
United States Geological Survey
2630 Fanta Reed Road
La Crosse, Wisconsin Â54603
www.umesc.usgs.gov
E-Mail: jknudson@xxxxxxxx
Phone: 608-781-6201
Cell: 608-304-4189


On Wed, May 10, 2017 at 10:24 AM, Todd Tannenbaum <tannenba@xxxxxxxxxxx> wrote:
On 5/10/2017 7:47 AM, Michael Di Domenico wrote:
Was there ever a resolution for this? I'm seeing the same issue with
8.6.1 on rhel7


Yes. I looked over Lee's (nomad) log files he sent me, and we discovered the culprit and a workaround for the few folks impacted.

Do you have NEGOTIATOR_HOST defined in your config file? I.e. if you do

 condor_config_val -v NEGOTIATOR_HOST

on any machine where condor_userprio is failing, does it show it being defined anywhere in your config file(s) ? If not, I am not sure what is going on and we will need to investigate. But if you do have NEGOTIATOR_HOST defined, here is the workaround to make things work again:

Â1. Remove the NEGOTIATOR_HOST setting from your config file(s).

Â2. Make sure your config file does not reference $(NEGOTIATOR_HOST) anywhere; if you do have such entries, replace $(NEGOTIATOR_HOST) with $(CONDOR_HOST).

References to NEGOTIATOR_HOST in the default condor_config you get when you install HTCondor fresh have been removed for years [1]... back 5 years ago we removed it from the default config and from the Manual, and attempted to purge it from the source code -- but somehow it landed back in the source code, perhaps due to IPv6 support...

At some point I suggest you "start fresh" by starting with a clean/default condor_config, and then just adding in your required custom changes. The "-writeconfig" option to condor_config_val could perhaps assist in this process.

Meanwhile we will attempt to purge NEGOTIATOR_HOST from the source again.

Hope this helps and apologies for the hassle,
regards,
Todd

[1] https://htcondor-wiki.cs.wisc.edu/index.cgi/tktview?tn=2929










On Tue, Mar 28, 2017 at 11:22 AM, Lee Damon <nomad@xxxxxxxxxxxxxxxxx> wrote:
Hi John,

I sent ToddT all the non-redacted information he requested. The thing
that stuck out at me was a statement that the IP address for the
negotiator is sinful. This says it's finding it just fine but objects to
something about the address itself. I'm waiting for Todd to have a
chance to look it over.

nomad

On 3/28/17 08:09 , John M Knoeller wrote:
what does condor_status -negotiator say?

The first thing condor_userprio does is try and fetch the negotiator ad from the collector so it can check
the negotiator version and see if it should talk to the collector or the negotiator for accounting information.

When it can't get a negotiator ad from the collector, this is the error it prints out.

You could also have a look in the collector log to see if the negotiator ad is being received. ÂPerhaps
the collector is refusing it because of your ALLOW_WRITE setting?

-tj

-----Original Message-----
From: HTCondor-users [mailto:htcondor-users-bounces@cs.wisc.edu] On Behalf Of Lee Damon
Sent: Monday, March 27, 2017 10:51 AM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: [HTCondor-users] condor_userprio: Can't locate negotiator in local pool

We just updated our condor pool using the condor-8.6.1-1.el7.x86_64 RPM
and everything is working except condor_userprio is slightly unhappy:

~ [76] ; condor_userprio
condor_userprio: Can't locate negotiator in local pool

The command works when I give it the name of the CONDOR_HOST:

~ [77] ; condor_userprio -pool redacted
Last Priority Update:Â 3/27 08:41
             Effective ÂPriority ÂRes ÂTotal Usage Time
Since
User Name         Priority  Factor ÂIn Use (wghted-hrs) Last
Usage
------------------------ ------------ --------- ------ ------------
----------
ajaech@xxxxxxxxxxxxxxxxx  137201.95 Â1000.00  208  198133.14
<now>
------------------------ ------------ --------- ------ ------------
----------
Number of users: 1Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â208Â Â 198133.14
0+23:59


The config files haven't changed - CONDOR_HOST and NEGOTIATOR_HOST
appear to be properly set:

~ [78] ; grep ^CONDOR_HOST /etc/condor/condor_config
CONDOR_HOSTÂ Â Â Â Â Â Â= redacted.ee.washington.edu
~ [79] ; grep ^NEGOTIATOR_HOST /etc/condor/condor_config
NEGOTIATOR_HOST = $(CONDOR_HOST)

condor_config_val(1) confirms these settings are applied.

Any hints on how I can fix this would be appreciated.
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxx.edu with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/



--
Todd Tannenbaum <tannenba@xxxxxxxxxxx> University of Wisconsin-Madison
Center for High Throughput Computing ÂDepartment of Computer Sciences
HTCondor Technical Lead        1210 W. Dayton St. Rm #4257
Phone: (608) 263-7132Â Â Â Â Â Â Â Â Â Madison, WI 53706-1685
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxx.edu with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/