Re: [HTCondor-users] optional input/output files

[Klint Gore <kgore4@xxxxxxxxxx>]

Unfortunately, the instance that caused the question is a directory and there's a lot more files the program creates in the top directory.

Would using a wrapper script with condor_chirp to change transfer_output_files work?  If I'm doing a wrapper script, I could probably just tar the output files at the cost of extra disk space.

#!/bin/bash

analysis_program

if [ -d optfolder ]

then

     condor_chirp set_job_attr transfer_ouput_files "resultsfolder,optionalfolder"

fi

klint.

---

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> on behalf of Jaime Frey via HTCondor-users <htcondor-users@xxxxxxxxxxx>

Sent: Saturday, 20 January 2024 2:54 AM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Cc: Jaime Frey <jfrey@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] optional input/output files

 

If you leave transfer_output_files unset, then all files in the jobâs working directory that were created or modified while the job ran are transferred back (files in sub-directories are ignored).

 - Jaime

On Jan 18, 2024, at 5:19âPM, Klint Gore <kgore4@xxxxxxxxxx> wrote:

Is there a way to mark entries in transfer_input_files and transfer_output_files as optional?

I've worked around transfer_input_files by using "include command" and a shell script to strip any that don't exist (probably over-thinking it).  Nothing obvious comes to mind for output files.

klint.