[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Condor-users] Too many open files
We're getting errors in the job log files indicating that there are too many files open:
007 (196430.005.000) 11/03 08:13:00 Shadow exception!
Error from slot12@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
: Failed to open '/mnt/render/jobs/job_141798_rndrgatebegin_yko_120_0400_syanye/chr_all_rp_tcrender-196430-5-stdout.txt' as standard output: Too many open files (errno 24)
0 - Run Bytes Sent By Job
0 - Run Bytes Received By Job
The file it's complaining about is the stdout from the job's executable. I've taken a look at the submit/scheduler machine and we're nowhere near the file limit. Same thing on the execution machine. We are however logging to a Windows share mounted to the submit/scheduler machine over CIFS. We've been experiencing extremely heavy load on the windows filer that we're logging to so I'm guessing it's a result of that but I wanted to throw this out there in case anyone else has run into similar issues before.