RE: [condor-users] Condor, compared to Sun Grid Engine

["Matt Hope" <Matt.Hope@xxxxxxxx>]

Are you in a homogenous job type environment?

If you only use the farm for blast jobs then the simple solution for now is set the Num VM's to 1. hey presto all is well.

This is obviously no use if the first question's answer is "No"
 
You could achieve something similar to the desired behaviour by meddling with the VM types and numbers per box, essentially providing one VM with all resources of which only one is allowed and then 2 or 4 normal vm's (one per real / logial CPU).
However preventing multiple jobs requiring different VM's running is time consuming and optimal allocation becomes nasty (and probably NP Complete) so don't expect utilization to be the best possible...

You can work round the utilization issues by exploiting any volume / time properties of your blast jobs. Alternatively split your farm on various vm types...

There was a very good post on this a short while ago I suggest you search the archives.

In todays cheaper multi CPU environments (not to mention 'hyper threading' and 'Cell' related technologies which further complicate the issue) it would be nice if condor could more effectively partition a nodes available resources in non uniform chunks.
I appreciate it's a harder problem to solve but is certainly something I think condor will need in future - condor on a PS3 anyone...

-----Original Message-----
From: owner-condor-users@xxxxxxxxxxx
[mailto:owner-condor-users@xxxxxxxxxxx]On Behalf Of Miskell, Craig
Sent: 19 April 2004 01:25
To: condor-users@xxxxxxxxxxx
Subject: [condor-users] Condor, compared to Sun Grid Engine


Hi,
	I'm evaluating software to use to control our compute farm, and
I'm currently tossing up between Sun Grid Engine, and Condor.  My gut
instinct is to like Condor more, probably due to it's roots - academic
rather than commercial, and it just "feels" right.  Plus, among other
things, SGE requires users have an account on each machine that they
want to execute a job on, whereas condor will use 'nobody' if required -
I much prefer condor's method as end users should never be able to even
theoretically login to our compute farm nodes.

But, SGE can do one particular thing that I'm not sure Condor can, and
this is a relatively important requirement for what we want to do with
our farm.  So, I thought I'd ask the list and see if there's some way I
can poke Condor into doing it too:

One of the tasks we will be using the compute farm for is Blast runs (a
bioinformatics thing).  We have some nice dual-proc Xeons with
hyperthreading which condor rightly interprets as 4 virtual
machines/node.  Empirical testing has shown that our blast runs go best
when a single job is told it can use 4 threads, which fully utilises the
node.  However, convincing condor to support this is non-obvious (to
me).  If I kick off 4 blast jobs, what I want to happen is condor to use
4 separate nodes.  Out of the box, what it'll do is use 4 vms on a
single node, which will slow blast down to a crawl.  I *could* space out
the starting of the runs so that the load average is up to 4 on each
node before I kick off another job, but that will introduce quite some
delay, and is a little bit crude for my liking.  SGE allows for
consumable resources, so I could setup a virtual "Blast" resource with a
count of 1 on each node, and then each blast job would "consume" that
blast resource (a "licence", if you will).  Is there anyway to do the
equivalent with Condor?  Perhaps advertise BLAST=1 for completely empty
nodes and the blast job sets off a script which changes the
advertisement for the node to BLAST=0, and require blast=1 for a blast
job to start?  Would the change in advert get back to the master before
the next job is dispatched to the same node?

Am I even looking at this the right way, or do I need to shift my view
completely?

I appreciate any help you can provide,

Thanks,

Craig Miskell,
Technical Support,
AgResearch Invermay
03 489-9279
I stopped and considered for a moment whether such a person 
would behave any differently  with his head cut right off, but 
then realized it would make a difference: it would allow  him to 
stand upright again.  	-- Anthony de Boer 
=======================================================================
Attention: The information contained in this message and/or attachments
from AgResearch Limited is intended only for the persons or entities
to which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipients is prohibited by AgResearch
Limited. If you have received this message in error, please notify the
sender immediately.
=======================================================================
Condor Support Information:
http://www.cs.wisc.edu/condor/condor-support/
To Unsubscribe, send mail to majordomo@xxxxxxxxxxx with
unsubscribe condor-users <your_email_address>



*****************************************************************
Gloucester Research Limited believes the information 
provided herein is reliable. While every care has been 
taken to ensure accuracy, the information is furnished 
to the recipients with no warranty as to the completeness 
and accuracy of its contents and on condition that any 
errors or omissions shall not be made the basis for any 
claim, demand or cause for action.
*****************************************************************

Condor Support Information:
http://www.cs.wisc.edu/condor/condor-support/
To Unsubscribe, send mail to majordomo@xxxxxxxxxxx with
unsubscribe condor-users <your_email_address>