[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Notification of multiple job completion

Have you thought of a workflow system, such as Pegasus/DAGMan (http://pegasus.isi.edu/), or Swift (http://www.ci.uchicago.edu/swift/)? These systems would take care of all data dependencies, and allow computations to continue after all needed data was available. 
Ioan Raicu, Ph.D.
Assistant Professor
Computer Science Department
Illinois Institute of Technology
10 W. 31st Street
Stuart Building, Room 237D
Chicago, IL 60616
Cel:    1-847-722-0876
Office: 1-312-567-5704
Email:  iraicu@xxxxxxxxxx
Web:    http://www.cs.iit.edu/~iraicu/

On 10/29/2010 4:24 PM, Rob Matthews wrote:



I am new to Condor and am using it for Monte Carlo simulation. Each MC run is independent and carried out by a given executable which produces a results file, so I have a wrapper program which populates the inputs for these and submits all the needed runs to the Condor queue. This all works great except now I need some way of knowing when all the MC runs I submitted are complete so I can postprocess results (i.e. parse all the individual results files and operate as needed).


Right now my wrapper code does this by polling the local directory every 5 seconds looking for the needed results files but this becomes inefficient with large simulations. Is there a mechanism in Condor to possibly execute a program (like my postprocessing code) once all the jobs submitted to the queue are compete?



Rob Matthews  



_______________________________________________ Condor-users mailing list To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a subject: Unsubscribe You can also unsubscribe by visiting https://lists.cs.wisc.edu/mailman/listinfo/condor-users The archives can be found at: https://lists.cs.wisc.edu/archive/condor-users/