[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Condor-users] any way to dynamically submit dependent jobs?

hi condor users,

greetings to the list. i have a use case that i'm struggling to fit into
condor. i'm a condor novice and i hope this is not an rtfm-type
question. i have a sequence of jobs. they must be executed in a certain
order. so far so good, but the complication arises in that the job
sequence is not fully-formed when the first job is submitted. this is
because the process of generating each job takes a significant amount of
time, and the details of subsequent jobs such as their Requirements are
unknown until they are generated. to maximize efficiency i would like to
submit a job A so that it can run while i'm generating a job B. the idea
is that when i'm done generating B, i would submit it under the
condition that it can't start until A finishes (if A is still running).
i am aware of dagman, but as far as i can tell it does not satisfy the
use case as the complete graph must be defined prior to A's submit time.
am i crazy or missing the obvious? any advice is greatly appreciated.