[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Condor-users] DAGman: array dependencies, dynamic DAGs



Hi folks, I just joined the list, and want to introduce myself. I was (I think) the first Condor user in the visual effects world back in 1998. I haven't been involved in Condor since 2004, but I'm back... and wow, you guys have been busy! I'm now at a small VFX studio in Montreal... we've jumped ship from a proprietary grid solution and I'm again looking at integrating Condor into a VFX workflow. I see consumable resources are now supported natively... cool. I have a couple questions about DAGman, though:

First, does DAGman have a way to declare dependencies between clusters of equal numbers of jobs, but have the dependencies actually be between jobs? Sun Grid Engine calls this an "array dependency". For example, given cluster A of 100 jobs, and cluster B of 100 jobs, I want to define a relationship between A and B such that:
 B.0 depends on A.0
 B.1 depends on A.1
 B.2 depends on A.2
 ...
 B.99 depends on A.99

This obviously requires that either A and B have the same number of jobs, or that the dependencies between A's jobs and B's jobs are explicitly described.

Second question: Is there any way to add or modify dependencies after a DAG is submitted? For example, I submit long-running cluster A, then several hours later I want to submit cluster B, such that B depends on the successful completion of A. Does DAGman re-read its dag file, or is it read only once?

cheers,
-Mark

P.S. Hey Todd!

--
Mark Visser, Software Director
Lumière VFX
Email: markv@xxxxxxxxxxxxxx
Phone: +1-514-316-1080 x3030