[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Condor-users] DC_AUTHENTICATE: attempt to open invalid session



Hi

Could someone explain this error message?

DC_AUTHENTICATE: attempt to open invalid session

It occurs every 5 minutes that there are jobs qued but no jobs run.

007.001:  Run analysis summary.  Of 5 machines,
     0 are rejected by your job's requirements
     0 reject your job because of their own requirements
     0 match, but are serving users with a better priority in the pool
     5 match, match, but reject the job for unknown reasons


I'm using $CondorVersion: 6.6.11 Mar 23 2006 $ $CondorPlatform: I386-LINUX_RH9 $ on all machines and therefore presume that the 6.4.X issues I read about on other threads aren't the problem.

I've ensured that all the machines have the same passwd for user condor and then restarted all the demonds.

Logs are below.  Thanks for any light you can shed.

John

[root@1XX sbin]# more ../local.1XX/log/*
::::::::::::::
../local.1XX/log/CollectorLog
::::::::::::::
5/30 08:28:59 ******************************************************
5/30 08:28:59 ** condor_collector (CONDOR_COLLECTOR) STARTING UP
5/30 08:28:59 ** /home/condor/sbin/condor_collector
5/30 08:28:59 ** $CondorVersion: 6.6.11 Mar 23 2006 $
5/30 08:28:59 ** $CondorPlatform: I386-LINUX_RH9 $
5/30 08:28:59 ** PID = 21291
5/30 08:28:59 ******************************************************
5/30 08:28:59 Using config file: /home/condor/etc/condor_config
5/30 08:28:59 Using local config files: /home/condor/local.1XX/condor_config.local
5/30 08:28:59 DaemonCore: Command Socket at <XXX.XXX.XXX.194:9618>
5/30 08:28:59 In ViewServer::Init()
5/30 08:28:59 In CollectorDaemon::Init()
5/30 08:28:59 In ViewServer::Config()
5/30 08:28:59 In CollectorDaemon::Config()
5/30 08:28:59 enable: Creating stats hash table
5/30 08:28:59 (Sent 0 ads in response to query)
5/30 08:28:59 Got QUERY_STARTD_PVT_ADS
5/30 08:28:59 (Sent 0 ads in response to query)
5/30 08:28:59 WARNING:  No master ad for < 1XX.XXX.XXX.ac.uk >
5/30 08:28:59 ScheddAd : Inserting ** "< 1XX.XXX.XXX.ac.uk , XXX.XXX.XXX.194 >" 5/30 08:28:59 stats: Inserting new hashent for 'Schedd':'1XX.XXX.XXX.ac.uk':'XXX.XXX.XXX.194'
5/30 08:29:04 ** Master < 1XX.XXX.XXX.ac.uk > rejuvenated from recently down
5/30 08:29:04 stats: Inserting new hashent for 'Master':'1XX.XXX.XXX.ac.uk':'XXX.XXX.XXX.194'
5/30 08:29:05 WARNING:  No master ad for < 3XX.XXX.XXX.ac.uk >
5/30 08:29:05 ScheddAd : Inserting ** "< 3XX.XXX.XXX.ac.uk , XXX.XXX.XXX.57 >" 5/30 08:29:05 stats: Inserting new hashent for 'Schedd':'3XX.XXX.XXX.ac.uk':'XXX.XXX.XXX.57'
5/30 08:29:10 ** Master < 3XX.XXX.XXX.ac.uk > rejuvenated from recently down
5/30 08:29:10 stats: Inserting new hashent for 'Master':'3XX.XXX.XXX.ac.uk':'XXX.XXX.XXX.57' 5/30 08:29:10 StartdAd : Inserting ** "< 1XX.XXX.XXX.ac.uk , XXX.XXX.XXX.194 >" 5/30 08:29:10 stats: Inserting new hashent for 'Start':'1XX.XXX.XXX.ac.uk':'XXX.XXX.XXX.194' 5/30 08:29:10 StartdPvtAd : Inserting ** "< 1XX.XXX.XXX.ac.uk , XXX.XXX.XXX.194 >" 5/30 08:29:10 stats: Inserting new hashent for 'StartdPvt':'1XX.XXX.XXX.ac.uk':'XXX.XXX.XXX.194'
5/30 08:29:11 WARNING:  No master ad for < 2XX.XXX.XXX.ac.uk >
5/30 08:29:11 ScheddAd : Inserting ** "< 2XX.XXX.XXX.ac.uk , XXX.XXX.XXX.58 >" 5/30 08:29:11 stats: Inserting new hashent for 'Schedd':'2XX.XXX.XXX.ac.uk':'XXX.XXX.XXX.58'
5/30 08:29:16 ** Master < 2XX.XXX.XXX.ac.uk > rejuvenated from recently down
5/30 08:29:16 stats: Inserting new hashent for 'Master':'2XX.XXX.XXX.ac.uk':'XXX.XXX.XXX.58'
5/30 08:29:22 WARNING:  No master ad for < vm1@xxxxxxxxxxxxxxxxx >
5/30 08:29:22 StartdAd : Inserting ** "< vm1@xxxxxxxxxxxxxxxxx , XXX.XXX.XXX.57 >" 5/30 08:29:22 stats: Inserting new hashent for 'Start':'vm1@xxxxxxxxxxxxxxxxx':'XXX.XXX.XXX.57' 5/30 08:29:22 StartdPvtAd : Inserting ** "< vm1@xxxxxxxxxxxxxxxxx , XXX.XXX.XXX.57 >" 5/30 08:29:22 stats: Inserting new hashent for 'StartdPvt':'vm1@xxxxxxxxxxxxxxxxx':'XXX.XXX.XXX.57'
5/30 08:29:23 WARNING:  No master ad for < vm2@xxxxxxxxxxxxxxxxx >
5/30 08:29:23 StartdAd : Inserting ** "< vm2@xxxxxxxxxxxxxxxxx , XXX.XXX.XXX.57 >" 5/30 08:29:23 stats: Inserting new hashent for 'Start':'vm2@xxxxxxxxxxxxxxxxx':'XXX.XXX.XXX.57' 5/30 08:29:23 StartdPvtAd : Inserting ** "< vm2@xxxxxxxxxxxxxxxxx , XXX.XXX.XXX.57 >" 5/30 08:29:23 stats: Inserting new hashent for 'StartdPvt':'vm2@xxxxxxxxxxxxxxxxx':'XXX.XXX.XXX.57'
5/30 08:29:30 WARNING:  No master ad for < vm1@xxxxxxxxxxxxxxxxx >
5/30 08:29:30 StartdAd : Inserting ** "< vm1@xxxxxxxxxxxxxxxxx , XXX.XXX.XXX.58 >" 5/30 08:29:30 stats: Inserting new hashent for 'Start':'vm1@xxxxxxxxxxxxxxxxx':'XXX.XXX.XXX.58' 5/30 08:29:30 StartdPvtAd : Inserting ** "< vm1@xxxxxxxxxxxxxxxxx , XXX.XXX.XXX.58 >" 5/30 08:29:30 stats: Inserting new hashent for 'StartdPvt':'vm1@xxxxxxxxxxxxxxxxx':'XXX.XXX.XXX.58'
5/30 08:29:31 WARNING:  No master ad for < vm2@xxxxxxxxxxxxxxxxx >
5/30 08:29:31 StartdAd : Inserting ** "< vm2@xxxxxxxxxxxxxxxxx , XXX.XXX.XXX.58 >" 5/30 08:29:31 stats: Inserting new hashent for 'Start':'vm2@xxxxxxxxxxxxxxxxx':'XXX.XXX.XXX.58' 5/30 08:29:31 StartdPvtAd : Inserting ** "< vm2@xxxxxxxxxxxxxxxxx , XXX.XXX.XXX.58 >" 5/30 08:29:31 stats: Inserting new hashent for 'StartdPvt':'vm2@xxxxxxxxxxxxxxxxx':'XXX.XXX.XXX.58' 5/30 08:30:03 SubmittorAd : Inserting ** "< ann@localhost , XXX.XXX.XXX.194 >" 5/30 08:30:03 stats: Inserting new hashent for 'Submittor':'ann@localhost':'XXX.XXX.XXX.194'
5/30 08:30:03 (Sent 12 ads in response to query)
5/30 08:30:03 Got QUERY_STARTD_PVT_ADS
5/30 08:30:03 (Sent 5 ads in response to query)
5/30 08:32:02 DC_AUTHENTICATE: attempt to open invalid session 1XX:22696:1148643645:9, failing. 5/30 08:32:06 DC_AUTHENTICATE: attempt to open invalid session 1XX:22696:1148643661:11, failing. 5/30 08:32:07 DC_AUTHENTICATE: attempt to open invalid session 1XX:22696:1148643661:11, failing. 5/30 08:32:10 DC_AUTHENTICATE: attempt to open invalid session 1XX:22696:1148643649:10, failing.
5/30 08:35:03 (Sent 12 ads in response to query)
5/30 08:35:03 Got QUERY_STARTD_PVT_ADS
5/30 08:35:03 (Sent 5 ads in response to query)
5/30 08:37:02 DC_AUTHENTICATE: attempt to open invalid session 1XX:22696:1148643645:9, failing. 5/30 08:37:06 DC_AUTHENTICATE: attempt to open invalid session 1XX:22696:1148643661:11, failing. 5/30 08:37:07 DC_AUTHENTICATE: attempt to open invalid session 1XX:22696:1148643661:11, failing. 5/30 08:37:10 DC_AUTHENTICATE: attempt to open invalid session 1XX:22696:1148643649:10, failing.
::::::::::::::
../local.1XX/log/MasterLog
::::::::::::::
5/30 08:28:58 ******************************************************
5/30 08:28:58 ** condor_master (CONDOR_MASTER) STARTING UP
5/30 08:28:58 ** /home/condor/sbin/condor_master
5/30 08:28:58 ** $CondorVersion: 6.6.11 Mar 23 2006 $
5/30 08:28:58 ** $CondorPlatform: I386-LINUX_RH9 $
5/30 08:28:58 ** PID = 21290
5/30 08:28:58 ******************************************************
5/30 08:28:58 Using config file: /home/condor/etc/condor_config
5/30 08:28:58 Using local config files: /home/condor/local.1XX/condor_config.local
5/30 08:28:58 DaemonCore: Command Socket at <XXX.XXX.XXX.194:43074>
5/30 08:28:59 Started DaemonCore process "/home/condor/sbin/condor_collector", pid and pgroup = 21291 5/30 08:28:59 Started DaemonCore process "/home/condor/sbin/condor_negotiator", pid and pgroup = 21292 5/30 08:28:59 Started DaemonCore process "/home/condor/sbin/condor_startd", pid and pgroup = 21293 5/30 08:28:59 Started DaemonCore process "/home/condor/sbin/condor_schedd", pid and pgroup = 21294
::::::::::::::
../local.1XX/log/MatchLog
::::::::::::::
::::::::::::::
../local.1XX/log/NegotiatorLog
::::::::::::::
5/30 08:28:59 ******************************************************
5/30 08:28:59 ** condor_negotiator (CONDOR_NEGOTIATOR) STARTING UP
5/30 08:28:59 ** /home/condor/sbin/condor_negotiator
5/30 08:28:59 ** $CondorVersion: 6.6.11 Mar 23 2006 $
5/30 08:28:59 ** $CondorPlatform: I386-LINUX_RH9 $
5/30 08:28:59 ** PID = 21292
5/30 08:28:59 ******************************************************
5/30 08:28:59 Using config file: /home/condor/etc/condor_config
5/30 08:28:59 Using local config files: /home/condor/local.1XX/condor_config.local
5/30 08:28:59 DaemonCore: Command Socket at <XXX.XXX.XXX.194:9614>
5/30 08:28:59 ACCOUNTANT_HOST = None (local)
5/30 08:28:59 NEGOTIATOR_INTERVAL = 300 sec
5/30 08:28:59 NEGOTIATOR_TIMEOUT = 30 sec
5/30 08:28:59 PREEMPTION_REQUIREMENTS = (CurrentTime - EnteredCurrentState) > (1 * (60 * 60)) && RemoteUserPrio > SubmittorPrio * 1.2 5/30 08:28:59 PREEMPTION_RANK = (RemoteUserPrio * 1000000) - TARGET.ImageSize
5/30 08:28:59 ---------- Started Negotiation Cycle ----------
5/30 08:28:59 Phase 1:  Obtaining ads from collector ...
5/30 08:28:59   Getting all public ads ...
5/30 08:28:59   Sorting 0 ads ...
5/30 08:28:59   Getting startd private ads ...
5/30 08:28:59 Got ads: 0 public and 0 private
5/30 08:28:59 Public ads include 0 submitter, 0 startd
5/30 08:28:59 Phase 2:  Performing accounting ...
5/30 08:28:59 Phase 3:  Sorting submitter ads by priority ...
5/30 08:28:59 Phase 4.1:  Negotiating with schedds ...
5/30 08:28:59 ---------- Finished Negotiation Cycle ----------
5/30 08:30:03 ---------- Started Negotiation Cycle ----------
5/30 08:30:03 Phase 1:  Obtaining ads from collector ...
5/30 08:30:03   Getting all public ads ...
5/30 08:30:03   Sorting 12 ads ...
5/30 08:30:03   Getting startd private ads ...
5/30 08:30:03 Got ads: 12 public and 5 private
5/30 08:30:03 Public ads include 1 submitter, 5 startd
5/30 08:30:03 Phase 2:  Performing accounting ...
5/30 08:30:03 Phase 3:  Sorting submitter ads by priority ...
5/30 08:30:03 Phase 4.1:  Negotiating with schedds ...
5/30 08:30:03   Negotiating with ann@localhost at <XXX.XXX.XXX.194:43076>
5/30 08:30:03     Got NO_MORE_JOBS;  done negotiating
5/30 08:30:03 ---------- Finished Negotiation Cycle ----------
5/30 08:35:03 ---------- Started Negotiation Cycle ----------
5/30 08:35:03 Phase 1:  Obtaining ads from collector ...
5/30 08:35:03   Getting all public ads ...
5/30 08:35:03   Sorting 12 ads ...
5/30 08:35:03   Getting startd private ads ...
5/30 08:35:03 Got ads: 12 public and 5 private
5/30 08:35:03 Public ads include 1 submitter, 5 startd
5/30 08:35:03 Phase 2:  Performing accounting ...
5/30 08:35:03 Phase 3:  Sorting submitter ads by priority ...
5/30 08:35:03 Phase 4.1:  Negotiating with schedds ...
5/30 08:35:03   Negotiating with ann@localhost at <XXX.XXX.XXX.194:43076>
5/30 08:35:03     Got NO_MORE_JOBS;  done negotiating
5/30 08:35:03 ---------- Finished Negotiation Cycle ----------
::::::::::::::
../local.1XX/log/SchedLog
::::::::::::::
5/30 08:28:59 ******************************************************
5/30 08:28:59 ** condor_schedd (CONDOR_SCHEDD) STARTING UP
5/30 08:28:59 ** /home/condor/sbin/condor_schedd
5/30 08:28:59 ** $CondorVersion: 6.6.11 Mar 23 2006 $
5/30 08:28:59 ** $CondorPlatform: I386-LINUX_RH9 $
5/30 08:28:59 ** PID = 21294
5/30 08:28:59 ******************************************************
5/30 08:28:59 Using config file: /home/condor/etc/condor_config
5/30 08:28:59 Using local config files: /home/condor/local.1XX/condor_config.local
5/30 08:28:59 DaemonCore: Command Socket at <XXX.XXX.XXX.194:43076>
5/30 08:30:03 IO: Failed to read packet header
5/30 08:30:03 DaemonCore: Command received via UDP from host <XXX.XXX.XXX.194:36995> 5/30 08:30:03 DaemonCore: received command 421 (RESCHEDULE), calling handler (reschedule_negotiator)
5/30 08:30:03 Sent ad to central manager for ann@localhost
5/30 08:30:03 Called reschedule_negotiator()
5/30 08:30:03 DaemonCore: Command received via TCP from host <XXX.XXX.XXX.194:43109> 5/30 08:30:03 DaemonCore: received command 416 (NEGOTIATE), calling handler (negotiate)
5/30 08:30:03 Negotiating for owner: ann@localhost
5/30 08:30:03 Checking consistency running and runnable jobs
5/30 08:30:03 Tables are consistent
5/30 08:30:03 Swap space estimate reached! No more jobs can be run!
5/30 08:30:03     Solution: get more swap space, or set RESERVED_SWAP = 0
5/30 08:30:03     0 jobs matched, 17 jobs idle
5/30 08:35:03 Sent ad to central manager for ann@localhost
5/30 08:35:03 Activity on stashed negotiator socket
5/30 08:35:03 Negotiating for owner: ann@localhost
5/30 08:35:03 Checking consistency running and runnable jobs
5/30 08:35:03 Tables are consistent
5/30 08:35:03 Swap space estimate reached! No more jobs can be run!
5/30 08:35:03     Solution: get more swap space, or set RESERVED_SWAP = 0
5/30 08:35:03     0 jobs matched, 17 jobs idle
::::::::::::::
../local.1XX/log/StartLog
::::::::::::::
5/30 08:28:59 ******************************************************
5/30 08:28:59 ** condor_startd (CONDOR_STARTD) STARTING UP
5/30 08:28:59 ** /home/condor/sbin/condor_startd
5/30 08:28:59 ** $CondorVersion: 6.6.11 Mar 23 2006 $
5/30 08:28:59 ** $CondorPlatform: I386-LINUX_RH9 $
5/30 08:28:59 ** PID = 21293
5/30 08:28:59 ******************************************************
5/30 08:28:59 Using config file: /home/condor/etc/condor_config
5/30 08:28:59 Using local config files: /home/condor/local.1XX/condor_config.local
5/30 08:28:59 DaemonCore: Command Socket at <XXX.XXX.XXX.194:43075>
5/30 08:29:00 New machine resource allocated
5/30 08:29:00 About to run initial benchmarks.
5/30 08:29:06 Completed initial benchmarks.
5/30 08:29:06 State change: IS_OWNER is false
5/30 08:29:06 Changing state: Owner -> Unclaimed
[root@1XX sbin]#

_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/