Re: [Condor-users] how to beef up the submit host ?

["Ian Chesal" <ICHESAL@xxxxxxxxxx>]

> Please pardon a few (likely) dumb questions from a newbie...

No problem. I'm happy to answer any questions you have.

> What are you using to do these things?

Our meta scheduler is build on the LAMP stack (that's Linux, Apache,
mod_perl, PostgreSQL for us). The Apache server offers up a SOAP
interface with a set of familiar methods: submit(), remove(), hold(),
query(), status(), etc. The user tool side uses a custom Cluster and Job
object that knows how to talk SOAP to the meta scheduler.

User interact with meta scheduler cluster and job ids, not condor
cluster and job ids. So when we submit from the meta scheduler to condor
we can break up a single user cluster into multiple condor clusters and
simply pause between condor submissions. The user interacts with one
"cluster" that maps to several condor clusters.

> What is it that "works great with Quill" ?

Quill is a Condor technology for storing the schedd Job ClassAds in a
PostgreSQL server. We've gone and added periodic pulls of the
condor_status command to the database, and our meta scheduler actually
queries the database directly and doesn't wrap the condor_q or
condor_status commands. Our schedd never actually see's a condor_q call
ever. Quill takes care of keeping our database up to date with all the
information condor_q gave us access to and it does it with zero stress
on the schedd.

> What is the meta-scheduler approach?

See my slides from the Condor Week 2006:

http://www.cs.wisc.edu/condor/CondorWeek2006/presentations/chesal_altera
.ppt

It's like "almost a scheduler" -- it doesn't do any heavy lifting, it
just passes work along. But it lets us scale and change the back end
without changing the front end.

- Ian