Matt
I am defiantly interested where can I find out more information on Rob Rati's implementation?
We have a client that submits jobs from windows to the condor cluster. The cluster is a all Linux machines RHEL 5.4.
What I learned this morning is that Ganglia was actually the root cause of the problem. I moved the master to a machine that did not have Ganglia and the RRDtool installed, and everything seems to work fine.
We are currently using a single generic queue for all the machines without any classads and using job hooks.
What I still find interesting is if I submit say 150 sleep jobs and I have 56 nodes available it seems that say 46 nodes will take off and run a batch then the number or running nodes drops to about half the started nodes then it will drop again until all the jobs complete the number of nodes continually declines in usage.
It just seems to be odd behavior I would expect all nodes to pick up and start working until all jobs were completed.
Thanks
Allen
-----Original Message-----
From: condor-users-bounces@xxxxxxxxxxx [mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of Matthew Farrellee
Sent: Tuesday, September 21, 2010 10:42 AM
To: Condor-Users Mail List
Subject: Re: [Condor-users] Condor with AMQP
Allen,
Are you using the Startd's work fetch (/job hooks) functionality to pull
work from the messaging queues?
If so, you should be looking at the execute nodes to see what the
problem may be. It might be helpful to describe the mechanism you're
using to deliver the work to Startds some more. Rob Rati implemented
just such a system on top of Qpid, which you might be interested in.
Best,
matt
On 09/20/2010 04:17 PM, Shahaan Ayyub wrote:
Allen,
I have never worked with Qpid but it seems from having a quick look at
the documentation that it simply provides a high level interface, in
your case, to condor. I am amazed as to why native condor commands are
not working? Otherwise you might have to look for a wrapper around
native condor commands.
Sorry couldn't be of much help to you.
Regards,
Shahaan
On 21/09/2010, at 5:03 AM, Shahaan Ayyub<shahaan@xxxxxxxxx
<mailto:shahaan@xxxxxxxxx>> wrote:
Hi Allen,
What does condor_q -better-analyze say for different timestamps, i.e.
when some of the jobs are held whilst some of them are still
running/completed.
Regards,
Shahaan
On 21/09/2010, at 3:07 AM, "Berg, Allen"<
<mailto:aberg@xxxxxxxx>aberg@xxxxxxxx<mailto:aberg@xxxxxxxx>> wrote:
We have a relatively small condor cluster its fifteen machines with a
total of 140 cpus.
We have implemented it using Apache Qpid Daemon is installed on the
master node. This package provides the queue “server”. It is the
facility that provides message queuing to the cluster. The Apache
Qpid API for C++ is installed on each cluster node.
What I am seeing that I have questions about is that when I submit
say two jobs very simple just a sleep command for two of the nodes.
The first job will take off and run, the second job will sit there
for possibly 20 minutes before it times out. Within any of the condor
logs I am not seeing any errors or any indications of weirdness. Then
if I run a larger test of say 40 jobs to sleep for 5 seconds, I would
expect that when I send the 40 jobs in they would all be picked up
and run completing in a reasonable amount of time. What I really see
is maybe 20 jobs take off, then 12 will start then maybe 8 and the
last few will complete. How can I find/learn out how the queue
actually performing and what can I do to better tune the queue.
Thanks
Allen
This message and any enclosures are intended only for the addressee. Please
notify the sender by email if you are not the intended recipient. If you are
not the intended recipient, you may not use, copy, disclose, or distribute this
message or its contents or enclosures to any other person and any such actions
may be unlawful. Ball reserves the right to monitor and review all messages
and enclosures sent to or from this email address.
_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to
<mailto:condor-users-request@xxxxxxxxxxx>condor-users-request@xxxxxxxxxxx
<mailto:condor-users-request@xxxxxxxxxxx> with a
subject: Unsubscribe
You can also unsubscribe by visiting
<https://lists.cs.wisc.edu/mailman/listinfo/condor-users>https://lists.cs.wisc.edu/mailman/listinfo/condor-users
The archives can be found at:
<https://lists.cs.wisc.edu/archive/condor-users/>https://lists.cs.wisc.edu/archive/condor-users/
_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/condor-users
The archives can be found at:
https://lists.cs.wisc.edu/archive/condor-users/
_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/condor-users
The archives can be found at:
https://lists.cs.wisc.edu/archive/condor-users/
This message and any enclosures are intended only for the addressee. Please
notify the sender by email if you are not the intended recipient. If you are
not the intended recipient, you may not use, copy, disclose, or distribute this
message or its contents or enclosures to any other person and any such actions
may be unlawful. Ball reserves the right to monitor and review all messages
and enclosures sent to or from this email address.
_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/condor-users
The archives can be found at:
https://lists.cs.wisc.edu/archive/condor-users/
![[Computer Systems Lab]](https://www-auth.cs.wisc.edu/pics/csl_logo.gif){kind=logo}