Re: [HTCondor-users] Transfer output files also on Job removal?

[Mark Coatsworth <coatsworth@xxxxxxxxxxx>]

Hi Oliver,

We have a little known knob that I think does exactly what you want.
Try adding the following to your submit file:

+SpoolOnEvict = false

I just did a test here and that seems to send the specified output
files back into my submit directory. Please give it a try and let us
know?

Mark

On Tue, Oct 20, 2020 at 4:17 PM Michael Pelletier via HTCondor-users
<htcondor-users@xxxxxxxxxxx> wrote:
>
> This came up in the 2019 HTCondor Week, and I discussed it a bit with the team, but I don't recall the specifics. Maybe one of them will chime in.
>
> Michael V Pelletier
> Principal Engineer
>
> Raytheon Technologies
> Information Technology
> Digital Transormation & Innovation
>
>
>
> -----Original Message-----
> From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Oliver Freyermuth
> Sent: Tuesday, October 20, 2020 7:27 AM
> To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
> Cc: Peter Wienemann <peter.wienemann@xxxxxxxxxxx>
> Subject: [External] [HTCondor-users] Transfer output files also on Job removal?
>
> Dear HTCondor experts,
>
> I've been looking for a knob, but could not find one for this use case:
> We'd love to transfer the output files of some jobs even in case of removal, i.e. condor_rm, or periodic_remove e.g. when a time limit is hit.
>
> The example use case here are interactive Jupyter notebooks: We don't want to share a file system with all the resources the notebooks may run on, and would love to use HTCondor file transfer to get things to and fro.
> However, interactive jobs are usually either cancelled by a time limit (i.e. periodic_remove), or (in case of JupyterHub's batchspawner) by condor_rm, so file transfer would be especially helpful for interactive use cases without shared file system.
>
> So it burns down to this:
>   Is there a way to also trigger file transfer on job removal (I have control over both schedd and submit file)?
>
>
> I have for now tried to set:
>   when_to_transfer_output = ON_EXIT_OR_EVICT
>   TransferOutputFiles = jupyter
>   ShouldTransferFiles = True
> and if I parse the logs correctly, indeed this triggers a file transfer to SPOOL on the schedd node, but then the files are purged afterwards, since the job was removed. Having them transferred back to the original place (or at least accessible) would be ideal in our case.
>
> Cheers and thanks in advance,
>         Oliver
>
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/



-- 
Mark Coatsworth
Systems Programmer
Center for High Throughput Computing
Department of Computer Sciences
University of Wisconsin-Madison