[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Condor-users] Notification of multiple job completion
- Date: Fri, 29 Oct 2010 16:50:08 -0500
- From: Cathrin Weiss <cweiss@xxxxxxxxxxx>
- Subject: Re: [Condor-users] Notification of multiple job completion
you may want to take a look at Condor DagMAN : With that you can specify dependencies between jobs (for example like in your case: run all jobs and then the postprocessor). That should help you a lot. (see: http://www.cs.wisc.edu/condor/manual/v7.4/2_10DAGMan_Applications.html )
On Oct 29, 2010, at 4:24 PM, Rob Matthews wrote:
> I am new to Condor and am using it for Monte Carlo simulation. Each MC run is independent and carried out by a given executable which produces a results file, so I have a wrapper program which populates the inputs for these and submits all the needed runs to the Condor queue. This all works great except now I need some way of knowing when all the MC runs I submitted are complete so I can postprocess results (i.e. parse all the individual results files and operate as needed).
> Right now my wrapper code does this by polling the local directory every 5 seconds looking for the needed results files but this becomes inefficient with large simulations. Is there a mechanism in Condor to possibly execute a program (like my postprocessing code) once all the jobs submitted to the queue are compete?
> Rob Matthews