Re: [Condor-users] any way to dynamically submit dependent jobs?

["Peter F. Couvares" <pfc@xxxxxxxxxxx>]

There's one important thing to note when using POST scripts to generate 
later nodes' submit files in a DAG -- since POST scripts may be 
executed more than once in case of failure, be careful that they are 
idempotent and do the right thing in that case (i.e., don't just append 
lines to an existing submit file, etc).

-Peter


On May 14, 2005, at 9:21 AM, Mark Silberstein wrote:

> Just as Peter wrote, you should have a static skeleton of the DAG 
> ready,
> but DAGman doesn't check the submission file until the job is actually
> submitted ( besides "log" parameter in .sub files, which by the way
> should be surrounded with spaces in order for DAGman to be able to read
> it correctly. It has some more subtleties, so read the manual about it)
> That means that you can simply regenerate the submission files of the
> DAG nodes which have not yet been submitted, during the DAG execution.
>
> I would use the following DAG:
>
> JOB A a.sub
> JOB GenerateB generate.sub
> JOB B B.sub
> SCRIPT PRE B regenerateB
> PARENT A GenerateB CHILD B
>
> If you say that generation of jobs takes significant amount of time, 
> you
> may want to do it in separate job: GenerateB
> Once A and GenerateB complete, PRE Script of B will regenerate B.sub
> according to the results of the two previous jobs. And then B will be
> submitted with the new submission file.
> Mark
--
Peter Couvares                        University of Wisconsin-Madison
Condor Project Research               Department of Computer Sciences
pfc@xxxxxxxxxxx                       1210 W. Dayton St. Rm #4241
(608) 265-8936                        Madison, WI 53706-1685