[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] timeout reading buffer



Dan,

I'd read about NEGOTIATOR_TIMEOUT and turned it up to 60, but it wasn't enough. Are there any formulas, so to speak, for setting a good value for it on a busy schedd?
Don't want to set it too high..

Didn't know about SCHEDD_TIMEOUT_MULTIPLIER, though, I'll try that, too..

Thanks,
-Preston

On Mar 2, 2006, at 3:21 PM, Dan Bradley wrote:


Preston,

I haven't looked at all of your reports in detail, but I'm guessing you may need to adjust some of the following timeouts if the schedd is not responding quickly enough to queries:

NEGOTIATOR_TIMEOUT
Sets the timeout that the negotiator uses on its network connections to the condor_ schedd and condor_ startds. It is defined in seconds and defaults to 30.

SCHEDD_TIMEOUT_MULTIPLIER
Set this to some integer (e.g. 2 or 10) to increase the timeouts that are used when communicating with the schedd.


--Dan

On Mar 2, 2006, at 1:44 PM, Preston Smith wrote:


On Mar 1, 2006, at 1:12 PM, Maxim Kovgan wrote:

Hi, Preston.

Qs:
* Are you using host based firewalls ?
  No.

* Can you look at /var/log/messages too ?

  Nothing syslogged besides gridftp connections.

* Are you using a good equipment (routers/switches) ?

  Yea. All my condor gear is directly connected into a Cisco 6509
core switch.
Cluster nodes are all on cisco 4948 leaf switches with 10 Gbit links
  back to said core switch.


* What is the topology of your network ?
  see above


I suspect the problem is either with OS or network, anyway, not
condor related.

  This schedd has been humming along busily for weeks, right up until
it got to
  about 3000 jobs queued up.

  The problem goes away when I hold half or so of the jobs in this
schedd.
  Now, with a large chunk of the queue held, condor's negotiated and
started
  hundreds of jobs like it should. I've got the queue drained by now,
though, just by
  holding a big chunk, and periodically releasing 6-700 jobs..

  So while I never really solved the problem, I've worked around it.

-Preston

--
Preston Smith  <psmith@xxxxxxxxxx>
Systems Research Engineer
Rosen Center for Advanced Computing, Purdue University



_______________________________________________
Condor-users mailing list
Condor-users@xxxxxxxxxxx
https://lists.cs.wisc.edu/mailman/listinfo/condor-users
_______________________________________________
Condor-users mailing list
Condor-users@xxxxxxxxxxx
https://lists.cs.wisc.edu/mailman/listinfo/condor-users

--
Preston Smith  <psmith@xxxxxxxxxx>
Systems Research Engineer
Rosen Center for Advanced Computing, Purdue University