[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] about different dagman jobs problem



On Thu, 1 Oct 2009, Tao.3.Chen@xxxxxxxxxxxxxxxxxxxxxxxxxxx wrote:

    I am using DAGMan to submit specific order jobs, I submit 2 different
dagman jobs to same executor PC(I have only one executor) :

dag1:
JOB testA testA.sub
JOB testB testB.sub
JOB testC testC.sub
PARENT testA testB CHILD testC

dag2:
JOB testD testD.sub
JOB testE testE.sub
JOB testF testF.sub
PARENT testD testE CHILD testF

But the result is: the executor will always execute
testA,testB,testD,testE,and then testC,testF, it is not what I want. I
need condor first finish dag1,then dag2,not like first all parent jobs
testA,testB,testD,testE, then child jobs testC,testF, does anyone have
this kind of problem? How can I solve this problem?
Thank you in advance!

Remember that there is no order of jobs guaranteed between separate DAGs.

Basically, you have two choices: make everything into one big DAG, or create a "parent" DAG that runs the two sub-DAGs.

If you make everything into once big DAG, you would do the following in
a single DAG file:

JOB testA testA.sub
JOB testB testB.sub
JOB testC testC.sub

JOB testD testD.sub
JOB testE testE.sub
JOB testF testF.sub

PARENT testA testB CHILD testC
PARENT testC CHILD testD testE
PARENT testD testE CHILD testF


If you want a parent DAG, you'd do the following:

dag_parent:

SUBDAG EXTERNAL first dag1
SUBDAG EXTERNAL second dag2
PARENT first CHILD second

(You would need to keep your existing dag1 and dag2, and only submit dag_parent.)

You can also use the SPLICE keyword, which works similarly to SUBDAG EXTERNAL in terms of dependencies, but runs everything under a single
DAGMan instance:

dag_parent
SPLICE first dag1
SPLICE second dag2
PARENT first CHILD second

(Note that the above discussion assumes that you're running a pretty recent Condor -- 7.3.x or late 7.2.x. If you have an older Condor version, you'll have to do things a bit differently.)

For more information, see sections 2.10.6.7 and 2.10.6.8 of the Condor manual:
http://www.cs.wisc.edu/condor/manual/v7.3/2_10DAGMan_Applications.html#SECTION003106700000000000000
http://www.cs.wisc.edu/condor/manual/v7.3/2_10DAGMan_Applications.html#SECTION003106800000000000000

Kent Wenger
Condor Team