Re: [Condor-users] Notification of multiple job completion

[Ioan Raicu <iraicu@xxxxxxxxxxxxxxx>]

Hi,
Have you thought of a workflow system, such as Pegasus/DAGMan (http://pegasus.isi.edu/), or Swift (http://www.ci.uchicago.edu/swift/)? These systems would take care of all data dependencies, and allow computations to continue after all needed data was available. 
Cheers,
Ioan

-- 
=================================================================
Ioan Raicu, Ph.D.
Assistant Professor
=================================================================
Computer Science Department
Illinois Institute of Technology
10 W. 31st Street
Stuart Building, Room 237D
Chicago, IL 60616
=================================================================
Cel:    1-847-722-0876
Office: 1-312-567-5704
Email:  iraicu@xxxxxxxxxx
Web:    http://www.cs.iit.edu/~iraicu/
=================================================================
=================================================================

On 10/29/2010 4:24 PM, Rob Matthews wrote:

Hi,

 

I am new to Condor and am using it for Monte Carlo simulation. Each MC run is independent and carried out by a given executable which produces a results file, so I have a wrapper program which populates the inputs for these and submits all the needed runs to the Condor queue. This all works great except now I need some way of knowing when all the MC runs I submitted are complete so I can postprocess results (i.e. parse all the individual results files and operate as needed).

 

Right now my wrapper code does this by polling the local directory every 5 seconds looking for the needed results files but this becomes inefficient with large simulations. Is there a mechanism in Condor to possibly execute a program (like my postprocessing code) once all the jobs submitted to the queue are compete?

 

Thanks!

Rob Matthews  

 

 

_______________________________________________
Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/condor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/condor-users/