[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Suppressing output to error log files



Just for everyone's information, I tried a "hack" but it didn't work:

1) Set "stream_output" to True.

2) Create a named pipe (fifo) in the file system.

3) Specify that the output be written to the named pipe.

4) Have another process read from the named pipe, do the filtering, and write to the "real" output file.


This didn't work because HTCondor tries to do a seek() on the specified file, which fails on a fifo.

Just letting everyone know so you don't duplicate effort. :)


Cheers,
-zach


> -----Original Message-----
> From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf
> Of Duncan Meacher
> Sent: Friday, July 21, 2017 8:49 AM
> To: htcondor-users@xxxxxxxxxxx
> Subject: [HTCondor-users] Suppressing output to error log files
> 
> Hi,
> 
> I was wondering if anyone knew of any way to suppress the output to the
> error (stderr) output log files, specified in the .sub file. At the moment
> we are using condor to run a gstreamer based job that is dealing with a few
> hundred streams and so any warning that appears will be printed a large
> number of times. From recent runs we were reached error log files of
> ~100MBs within 10 minutes, and this job needs to be run continuously.
> 
> When running from the command line we're able to to do some piping such as:
> 
> $ command  |& grep -v 'XLAL\|GSL\|Generic'
> 
> Is there anyway of repeating this process with condor?
> 
> Thanks, Duncan
> 
> 
> --
> 
> ==========================
> 
> Duncan Meacher, PhD
> Postdoctoral Researcher
> Institute for Gravitation and the Cosmos
> Department of Physics
> Pennsylvania State University
> 104 Davey Lab #040
> University Park, PA 16802
> Tel: +1 814 865 3243
> ==========================