[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [HTCondor-users] Determine when all jobs in a cluster have finished?
- Date: Wed, 30 Jan 2013 14:42:07 +0000
- From: Brian Candler <B.Candler@xxxxxxxxx>
- Subject: Re: [HTCondor-users] Determine when all jobs in a cluster have finished?
On Wed, Jan 30, 2013 at 09:21:46AM -0500, Brian Pipa wrote:
> I'd really like the whole thing to be self-contained in one DAG like:
> Job QueryDB querydb.job
> Job Workers workers.job
> Job PostProcess postprocess.job
> PARENT QueryDB CHILD Workers
> PARENT Workers CHILD PostProcess
> since that seems much simpler and self-contained but I don't think
> that's doable since the results of the QueryDB job determines the data
> and number of worker jobs I'll need.
I thought I read something along those lines a while back. In principle I
think it is doable, because workers.job is not processed until the time that
job is actually submitted.
I don't know the cleanest way of making this dynamic, but you could try
simply overwriting the workers.job file from the querydb job.
Maybe what I am thinking of is this:
It says explicitly that a DAG may modify a sub-DAG before it is submitted.