[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] timeout reading buffer




Preston,

I haven't looked at all of your reports in detail, but I'm guessing you may need to adjust some of the following timeouts if the schedd is not responding quickly enough to queries:

<x-tad-bigger>NEGOTIATOR_TIMEOUT
</x-tad-bigger>
Sets the timeout that the negotiator uses on its network connections to the condor_ schedd and condor_ startds. It is defined in seconds and defaults to 30.

SCHEDD_TIMEOUT_MULTIPLIER
Set this to some integer (e.g. 2 or 10) to increase the timeouts that are used when communicating with the schedd.


--Dan

On Mar 2, 2006, at 1:44 PM, Preston Smith wrote:

On Mar 1, 2006, at 1:12 PM, Maxim Kovgan wrote:

Hi, Preston.

Qs:
* Are you using host based firewalls ?
No.

* Can you look at /var/log/messages too ?

Nothing syslogged besides gridftp connections.

* Are you using a good equipment (routers/switches) ?

Yea. All my condor gear is directly connected into a Cisco 6509
core switch.
Cluster nodes are all on cisco 4948 leaf switches with 10 Gbit links
back to said core switch.


* What is the topology of your network ?
see above

I suspect the problem is either with OS or network, anyway, not
condor related.

This schedd has been humming along busily for weeks, right up until
it got to
about 3000 jobs queued up.

The problem goes away when I hold half or so of the jobs in this
schedd.
Now, with a large chunk of the queue held, condor's negotiated and
started
hundreds of jobs like it should. I've got the queue drained by now,
though, just by
holding a big chunk, and periodically releasing 6-700 jobs..

So while I never really solved the problem, I've worked around it.

-Preston

--
Preston Smith <psmith@xxxxxxxxxx>
Systems Research Engineer
Rosen Center for Advanced Computing, Purdue University



_______________________________________________
Condor-users mailing list
Condor-users@xxxxxxxxxxx
https://lists.cs.wisc.edu/mailman/listinfo/condor-users