[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Intel Core i7 processors?

On Tue, Apr 13, 2010 at 4:28 PM, Tanzima Zerin Islam <tz.islam@xxxxxxxxx> wrote:
Sounds interesting... Do you have any paper or some other documentation where this information has been published or something of that sort? I was interested to find out about the actual overhead that condor might add.... so any pointer to any publication will save me some time...

We have some of our benchmarking methodology published here: http://bit.ly/aqkYkw

Our noise margins are not part of that paper though. I doubt we'd ever do a full paper on this. Highly competitive industry, FPGAs, and there's really only us and them. So we tend not to say too much about things that enhance our productivity. :)

Condor itself doesn't add overhead. It's not intercepting any system calls or anything like that when you're using the vanilla universe. What it's doing is stealing CPU cycles. That's not what I've traditionally referred to as "overhead".

The daemons running on the machines need to do periodic work: pull more jobs, check in with the collector and update their ClassAd, etc. And that work Condor does steals a very minimal amount of CPU time away from your jobs. We run our 2x4 core boxes very hot: every job is multi-threaded, generally running 8 threads at the peak of its work life and we run 8 jobs in parallel with Condor just to give you an idea of how overloaded these machines are. Condor is able to do its thing and inject only that small amount of noise into the run time of our jobs.

Total noise margin for us, without Condor running our jobs, is in the  +/- 1-2% range, so under Condor's control it's about +/- 4%. That means a code change that shows a reduction of less than 4% in run time needs a *lot* of tests run to validate it is indeed making an improvement and we're not just seeing noise. Noise is highly dependent on your tool, so anything I did publish would be pretty specific to Quartus II (which just happens to be both IO bound and CPU bound depending on where you are in the CAD flow...fun).

- Ian