Re: [Condor-users] Condor's DAG scalability?

[Zachary Miller <zmiller@xxxxxxxxxxx>]

> 
> May I solicit some words of wisdom with respect to the scalability of 
> Condor's DAG functionality as I will be using it?  :-)  Have others used 
> Condor's DAG tools for single experiments in which there are thousands 
> (or even millions) of component processes?

i have, just for fun, run a dag with over a million jobs.  it takes a while
to start up and parse the whole .dag file, maybe up to an hour, but once it
gets started the overhead is minimal.  keep as much of dagman's stuff as
possible (like logs, submit files) out of NFS.  at that scale, something
*will* fail eventually and your dag may abort.


cheers,
-zach