[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Architechture suggestions for large Condor pool

      that is big. Have you seen the ppt from condor uk week:


It has some interesting info that may identify hotspots.

BTW how are you doing the authentication against AD for the windows
machines and linux machines?

Cheers Paul
Why should one ask everyone to reach one's own exalted heights and esoteric
grounds instead of bending down to the poor prune's level and showing him
the right way up in a reasonable way?  He has but to travel up slowly.  And
not all may reach all the way up. --Quasi

                      Dan Swan                                                                                                             
                      <dan.swan@xxxxxxxxx>         To:       condor-users@xxxxxxxxxxx                                                      
                      Sent by:                     cc:       (bcc: Paul Chubb/Staff/ABS)                                                   
                      condor-users-bounces@        Subject:  [Condor-users] Architechture suggestions for large Condor pool                
                      08/02/2005 11:55 PM                                                                                                  
                      Please respond to Dan                                                                                                
                      Swan; Please respond                                                                                                 
                      to Condor-Users Mail                                                                                                 

Hi all,

This is more of a general than technical question.  We are
investigating and planning the deployment of Condor to about 8,000 to
10,000 machines in  a mixed Linux/Windows environment.

The initial roll out will be to a number of Linux clusters to test the
technology, but eventually roll out will occur across several
departments.  I'm interested in how people have scaled up to handle
clusters in this range, my experience is limited to single submit
host/40 machines.. so this is a bit of a jump.

In terms of architechture what should we be thinking about right now?
Would people suggest a number of flocked condor pools each with a
submit host?  Would a single submit host fall over trying to handle
the needs of a potentially large number of on-site users?
We're also interested in access control and authentication setups on
such a large scale.  (Current authentication is via AD which the Linux
clusters also authenticate against).  The ability for certain
departments to have priority over their own hardware would be nice, or
limiting users to certain pools etc.

I really would be grateful for any pointers to documentation, contact
with administrators of other large Condor deployments, or just general
thoughts on setup strategy.  I know this is a wide open question, and
all feedback would be gratefully received.



Dan Swan || dan.swan[at]gmail.com || http://scot1and.net/~dan
"Reality is that which, when you stop believing in it, doesn't go away."
(Philip K. Dick - How to Build a Universe)
Condor-users mailing list

ABS Web Site:  www.abs.gov.au