[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Need some guidelines for a chunking up vanilla Win appplication across nodes



On Tue, 27 Feb 2007, Andrew Cunningham wrote:

> 	I have this all working great. It seems an obvious question, but the
> last step in my process is to run a program that collates the
> results.This cannot run until all jobs are complete and the output
> files are back at the submitting machine. How does one know this
> apart from watching condor_q

This seems like a case where DAGMan
(http://www.cs.wisc.edu/condor/manual/v6.8/2_11DAGMan_Applications.html)
would help.

You can have a simple DAG where your processing steps are all parents
of the collating step.

DAGMan will take care of watching the log files, you don't have to worry
about running lots of condor_qs, you get various error recovery
facilities, etc.

Kent Wenger
Condor Team