Re: [Condor-users] Too many popen() calls in DAGMan ?

["Matt Hope" <matthew.hope@xxxxxxxxx>]

On 9/6/06, Masakatsu Ito <m-ito@xxxxxxxxxxxxxx> wrote:
> Thank you for your prompt response, and advices.

no prob

> We use Linux PC to manage  condor  pool,  so  I check  its
> /proc/sys/kernel/threads-max . The maximum number of processes
> is 32474 .
>
> Is this too small for my simulations, where .dag file consists of 16341
> nodes
> and 396*2 jobs are submitted to CONDOR simultaneously?

For the shadows this should be fine. each running job uses one
process, so 128 in 32K is fine

> > As to the version you might want to take a look at the BugFixes in
> > http://www.cs.wisc.edu/condor/manual/v6.7/8_3Development_Release.html
> > to see if there is any thing about DAGMan you should know
>
> I find that version 6.7.19 of condor_dagman no longer uses
> the popen() system call.  So, if the usage of popen() causes
> the failure of job submissions,  the later version of DAGMan would
> complete my simulations.  But shadow daemon may eat up
> our process/thread limit (kernel.threads-max = 32474).
>
> hmmm, I'm a bit confused...  I'd appreciate more help or hints.

That sounds like it is the culprit. Upgrading to 6.7.19+ would be in order.

Matt