[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Condor-users] Condor based Meta Data Collector/Negotator for cross domain pooling

Dear condor gurus,

I'm new to the condor world as it is, but I've got a rather sizable task. I've seen some stuff on the mailing list that looks similar to what I need, but not enough to give me a clue on how to address it.

Basically, I have several sets of resources. I have stand alone work stations, that are organized as one pool, with a dedicated collector/negotiator. I have a cluster, that's on a private network, with one front-end, that's doing IP Masquerading, organized as a pool. The collector is listening on the public interface, and all the nodes report to that interface. On top of all of this, I also have external resources. to the best of my knowledge, we have no administrative domain with these resources, but they do accept our signing policy for x509 proxy certs, and they are all running both Condor and Globus.

SO......my challenge. I need to basically pool these resources all together. The "ideal view" would basically be to have one meta collector/negotiator, for lack of a better term. A single machine that can give stats, and do match making, even abstractly to the various pools and such. From the sound of things, there are bits and pieces of what I need around, but I haven't figured out how to get all the parts together. For example, I can pole a pool with something like, condor_status -pool collector-name.domain.edu. And I can set up flocking from this meta host to the various pools, but that doesn't solve my out of domain issue. I'd REALLY like to have a single, replicate-able, solution, so even if the local resources have a little more overhead and config work, it's fine, cause I'd rather support one variation on this configuration, rather than having a specific config for each pool that this meta host talks to. Thanx all for any advice, reading material, or others. Again, I have no shame admitting I'm a complete newbie, so really really small words, and simple questions won't be offensive.

So the Q/A part.

A.) Do I need to do anything further for the cluster pool, as long as IP Masquerading is working? I don't need to do explicit port forwarding or anything, right?

B.) Can I make a meta collector, that receives all of the ClassAds from the various pool collectors? Or should I make something that actively collects from each pool? This second option might accommodate the "external" pool issue best? (example condor_status - long -pool external.domain.edu)

C.) Assuming B, Can I leverage this meta collector to do some form of meta negotiation that's homogeneously managed across all these resources, even the external pool?

D.) Is this something that can be manageable from the central meta negotiator? Can jobs be started and stopped and such from the meta host, or will it not be feasible, since there are various schedd's running on the various distributed pools?

_______________________________________________________ Adam Lathers NCMIR: National Center for Microscopy and Imaging Research Network Systems Engineer phone: (858) 822-0735 fax: (858) 822-0828 web: http://ncmir.ucsd.edu