Re: [Condor-users] managing large numbers of a job's output and error files

[Nathan Panike <nwp@xxxxxxxxxxx>]

On Thu, May 31, 2012 at 12:43:34PM -0500, Stephen Pietrowicz wrote:
> Hi,
> 
> I have a dag that has 10,000 jobs associated with it.  It is submitted to a local condor pool with all disks on NFS.
> 
> The jobs all send output and error to their own files, using a variable I set.
> 
> output=logs/worker-$(var2).out
> error=logs/worker-$(var2).err
> 
> I can run the job, and it works fine, except that I end up with 20,000 files in one directory.   I'd like to have something like this:
> 
> output=logs/$(dir1)/$(dir2)/worker-$(var2).out
> error=logs/$(dir1)/$(dir2)/worker-$(var2).err
> 
> but when I tried a simple version of it, 
> 
> output=logs/pre/worker-$(var2).out
> 
> I got an error that says:
> 
> statfs(/home/srp/orca_scratch/srp12may30_46/logs/pre/worker-pre.log) failed: 2/No such file or directory
> 05/31/12 12:09:35 WARNING: can't determine whether log file home/srp/orca_scratch/srp12may30_46/logs/pre/worker-pre.log is on NFS.
> 05/31/12 12:09:35 DAGMan::Job:8001:ERROR: Unable to monitor log file for node A|ReadMultipleUserLogs:9004:Error getting file ID in monitorLogFile()|ReadMultipleUserLogs:9004:Error initializing log file /home/srp/orca_scratch/srp12may30_46/logs/pre/worker-pre.log|MultiLogFiles:9001:Error (2, No such file or directory) opening file  /home/srp/orca_scratch/srp12may30_46/logs/pre/worker-pre.log for creation or truncation
> 05/31/12 12:09:35 ERROR "Fatal log file monitoring error!
> " at line 858 in file /slots/05/dir_34706/userdir/src/condor_dagman/job.cpp
> 
> I already have to create the "logs" directory before running the job.  Is there a way around having to create the "logs" directory and all the subdirectories below it before submitting the job?  In other words, is there a way to make condor create the directory structure the .out and .err files are being written to?

Hello Steve:

No, condor will not create the directory.  But you could have a PRE
script for each DAGman job that creates the necessary directories 

Nathan Panike

> 
> Thanks for any info you can provide,
> 
> Steve