[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Condor-users] condor pools question
- Date: Fri, 6 Aug 2010 09:31:33 -0400
- From: "Berg, Allen" <aberg@xxxxxxxx>
- Subject: Re: [Condor-users] condor pools question
Great thanks a lot
I was wondering about the pools as that section is not written yet. Are pools a new feature?
While I understand the ranking and sub-pools as you have described it I would think there might be a more dynamic way of assigning particular computer to a sub pool. I will give it a shot as suggested.
From: condor-users-bounces@xxxxxxxxxxx [mailto:condor-users-bounces@xxxxxxxxxxx] On Behalf Of David Kotz
Sent: Thursday, August 05, 2010 11:20 AM
To: Condor-Users Mail List
Subject: Re: [Condor-users] condor pools question
I saw your post yesterday, but I wasn't certain exactly what you're
trying to accomplish with your function distinctions. I'll make some
suggestions based on my guess that you're wanting to guarantee that at
least N number of slots is available for jobs of each type you mention.
Let's assume you have all machines pointing at the same central manager:
CONDOR_HOST = condormanager.ball.com
for instance, set in the main Condor config file, either in a shared
filesystem or copied to all hosts. All of the settings common to all of
your Condor worker nodes go in that file.
To differentiate between the various sub-pools, you can use local config
files, as defined in the main config file with
LOCAL_CONFIG_FILE = $(RELEASE_DIR)/etc/local/$(HOSTNAME)
REQUIRE_LOCAL_CONFIG_FILE = TRUE
I would write one config file for R&D, one for Train, one for Test, and
one for Prod. In the local config directory above, I'd have symlinks
pointing hostnames at those config files. To shift machines from one
sub-pool to another, you'd switch their symlinks to point to the
appropriate config file and restart the Condor daemons.
In each local config file, you'd use START and/or RANK expressions to
determine what kinds of jobs each sub-pool will run. You might base
this on the user account or maybe the machine used to submit the jobs.
See this section of the Condor docs:
On Thu, 2010-08-05 at 10:48 -0400, Berg, Allen wrote:
> We are thinking about using condor, I have a question first
> If I want to create a pool of servers for different functions for
> example R&D, Train, Test, Prod, so in other words I have say 2 servers
> for R&D and 15 in production and want to resize the pool, to have a
> few servers working in training for a class that’s being put on
> tomorrow. I would just add the servers via the collector_name =
> train. My understanding is the master node could serve all of the
> different “environments with no issue or further setup. Is that
> This message and any enclosures are intended only for the addressee. Please
> notify the sender by email if you are not the intended recipient. If you are
> not the intended recipient, you may not use, copy, disclose, or distribute this
> message or its contents or enclosures to any other person and any such actions
> may be unlawful. Ball reserves the right to monitor and review all messages
> and enclosures sent to or from this email address.
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> The archives can be found at:
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
You can also unsubscribe by visiting
The archives can be found at:
This message and any enclosures are intended only for the addressee. Please
notify the sender by email if you are not the intended recipient. If you are
not the intended recipient, you may not use, copy, disclose, or distribute this
message or its contents or enclosures to any other person and any such actions
may be unlawful. Ball reserves the right to monitor and review all messages
and enclosures sent to or from this email address.