Re: [HTCondor-users] Suppressing output to error log files

[Duncan Meacher <duncan.meacher@xxxxxxx>]

Hi all,

Thanks for checking and getting back to me. Yeah, a separate process to cleanup the output files wont be a problem to set up, I just wanted to see if there was any workaround first.

Thanks, Duncan

On 1 August 2017 at 13:17, Brian Bockelman <bbockelm@xxxxxxxxxxx> wrote:

Hi Duncan,

I think your best bet is going to be to wrap the entire job in a script, and do the filtering in bash.

So, instead of your command being the python (or whatever) script directly, it'll call a bash script that then invokes python.

Disappointing - but probably the most expedient way forward.

Brian

> On Jul 21, 2017, at 8:48 AM, Duncan Meacher <duncan.meacher@xxxxxxx> wrote:
>
> Hi,
>
> I was wondering if anyone knew of any way to suppress the output to the error (stderr) output log files, specified in the .sub file. At the moment we are using condor to run a gstreamer based job that is dealing with a few hundred streams and so any warning that appears will be printed a large number of times. From recent runs we were reached error log files of ~100MBs within 10 minutes, and this job needs to be run continuously.
>
> When running from the command line we're able to to do some piping such as:
>
> $ command |& grep -v 'XLAL\|GSL\|Generic'
>
> Is there anyway of repeating this process with condor?
>
> Thanks, Duncan
>
> --
> ==========================
>
> Duncan Meacher, PhD
> Postdoctoral Researcher
> Institute for Gravitation and the Cosmos
> Department of Physics
> Pennsylvania State University
> 104 Davey Lab #040
> University Park, PA 16802
> Tel: +1 814 865 3243
> ==========================

> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@cs.wisc.edu with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@cs.wisc.edu with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

--

==========================

Duncan Meacher, PhD
Postdoctoral Researcher
Institute for Gravitation and the Cosmos
Department of Physics
Pennsylvania State University
104 Davey Lab #040
University Park, PA 16802
Tel: +1 814 865 3243
==========================