[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Job TMP environment variable



On 12/22/20 5:33 PM, Todd Tannenbaum wrote:
> On 12/22/2020 6:14 AM, Petr Vokac wrote:
>> MOUNT_UNDER_SCRATCH is exactly the reason why I would like to have
>> all TMP variants pointing to /tmp. It seems to me HTCondor always set
>> these environment variables to the job working directory
>>
>
> Could you help me understand the motivation for wanting TMP to go to
> /tmp ?
>
> My guess is that it is not really related to MOUNT_UNDER_SCRATCH, but
> that you want all the job's temporary files in a subdirectory of the
> _CONDOR_SCRATCH_DIR so that they will not get automatically included
> in transfer_output_files. It just so happens that MOUNT_UNDER_SCRATCH
> does this (makes a subdirectory for tmp files in the scratch dir). Is
> my guess correct or is there some other motivation I have overlooked?

Yes, you are right ... my motivation is ARC-CE behavior that rely on
HTCondor mechanism for automatic transfer of output files created in
_CONDOR_SCRATCH_DIR (no transfer_output_files defined in job submission
file ... this is sufficient for simple ARC-CE outputs used by common
grid workflows), but in case job use TMP environment variable to store
(huge) temporary junk files than these files are automatically
transferred back by HTCondor. It seems to me more natural to use /tmp
mapped in ${_CONDOR_SCRATCH_DIR}/tmp for temporary files that should not
be considered as a job output.

Petr