[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Is DAGMan the answer to my question?

On Thu, 19 Feb 2009, Hoover Sam wrote:

I want to run a large batch of jobs, lets say around 10,000 on my private condor pool, then have a single email notification sent when all 10,000 jobs are finished. I do not want to get 10,000 emails, one for each job that completes.

The number of jobs changes, I build the condor.submit files on the fly and submit them all at once after the submit files are built.

I could see building a single jobfile.dag file with the 10,000 jobs as parent, then just one child to send an email notification, but was hoping for a simpler solution. Perhaps this is the simplest solution, but I'm new to Condor so I thought I'd ask.

There is a simpler solution -- if you make such a DAG, you'll get a notification email (by default) when the DAGMan job itself finishes. So you don't need to add any "extra" job just to get the email.

(Also, in your DAG file, don't specify any parent/child relationships -- I'm assuming that you want all 10,000 jobs to run in parallel as much as possible, and you're just using DAGMan for the email notification.)

Kent Wenger
Condor Team