[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[condor-users] easily submitting many identical jobs with DAGMan (was Re: Dagman job submission problem)
- Date: Thu, 11 Dec 2003 12:05:21 -0600
- From: "Peter F. Couvares" <pfc@xxxxxxxxxxx>
- Subject: [condor-users] easily submitting many identical jobs with DAGMan (was Re: Dagman job submission problem)
Antti Eskola wrote:
> I just found out that dagman does not support submitting many jobs
> in one submit file... :(
True -- but this is by design, and shouldn't be a problem, as I'll
> but if anybody has any idea pls let me know.
> Next thing is test making 0...n submit files and a really big dagman
> submit file :) fearing that it wont work when manymany jobs...
Although DAGMan (for important reasons) does not allow you to use the
Condor "queue N" feature to submit N instances of the same job from
one submit file, you *can* still use a single, common submit file for
multiple nodes in a DAG. E.g.:
Job A submit.cmd
Job B submit.cmd
Job C submit.cmd
As long as the submit.cmd file is written like it would be for use
with the "queue N" feature, in that each instance of the job has
unique input/output/error files (e.g., by using the $(Cluster) macro),
then it is not necessary to make N copies of the same submit file in
order to submit N identical jobs in a DAG.
This should be just as convenient as using "queue N" -- you still only
need one submit file -- but it results in a well-defined DAG in which
each Condor job is explicitly declared as a node in the DAG, and there
is only one Condor job per DAG node.
This one-to-one relationship is important for the DAG semantics. If
"queue N" were allowed and thus more than one Condor job could exist
for a single DAG node, then a node would no longer be atomic, the DAG
semantics would be violated, and recovery, etc. would be complicated
I hope this helps. Let me know if you have other questions.
Peter Couvares http://www.cs.wisc.edu/~pfc
Condor Project Research pfc@xxxxxxxxxxx
University of Wisconsin-Madison (608) 265-8936
Condor Support Information:
To Unsubscribe, send mail to majordomo@xxxxxxxxxxx with
unsubscribe condor-users <your_email_address>