Re: [Condor-users] Benchmarking condor

[Matt Hope <matthew.hope@xxxxxxxxx>]

On 6/30/05, Juan Ignacio Sánchez Lara <juanignaciosl@xxxxxxxxx> wrote:
> Hello, Matt
> 
> 
> > Take a look at the sum of:
> > condor_status -format "vm%d@" VirtualMachineId -format "%s " Machine 
> > -format "%d " KFlops -format "%d\n" Mips
> > 
> > The KFlops/MIPS values are about as useful as BogoMIPS et al. i.e.
> > useful only as a vague indicator of relative raw CPU performance.
> > 
> > The best way to benchmark your pool is to take the suite of 
> > applications you run on it and run each one in some well defined,
> > repeatable and close to reality mode then see how long the jobs take
> > to finish.
> 
> This is my actual problem. As you say, Mips, Flops and so on only measure
> CPU power, and that's not that important. From my point of view the best
> measure would be one which said something like "if you run your bunch of
> software in our cluster instead of your computer you'll do X times faster.
> Obviously the best (and maybe only) way to do this is actually running the
> software.

If you are seeking to convince people to run on the pool rather than
locally then a good way is:
Take a few people who are reasonably technically literate, hopefully
with good relationships to other people/groups. Do some direct work
with them to help them get their stuff working on the farm and get
some real quantifiable speed ups from them. This will give you the
ability to say:

"Team|Person X saves Y% on their jobs"

the affected person will hopefully also sing the praises of how much
quicker they can get stuff done...before you know it you're away. And
with some good quantifiable numbers to boot.

The biggest concern for most jobs is making them parallel such that
the number of jobs exceeds the number of nodes (so there is more room
for scaling the farm by adding nodes).

Matt