[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [HTCondor-users] Do jobs perform/require a successful checksum before they begin?
- Date: Tue, 31 Mar 2020 14:11:51 +0000
- From: Jaime Frey <jfrey@xxxxxxxxxxx>
- Subject: Re: [HTCondor-users] Do jobs perform/require a successful checksum before they begin?
> On Mar 26, 2020, at 2:52 PM, Todd L Miller <tlmiller@xxxxxxxxxxx> wrote:
>> TLDR question is: Are files checksummed upon transfer before the job can start?
> No. You can force HTCondor to checksum the network transfer by enabling integrity (which may also required encryption for file transfers; see https://htcondor.readthedocs.io/en/latest/admin-manual/configuration-macros.html#SEC_*_INTEGRITY),
> but there's no end-to-end verification.
I believe HTCondorâs normal integrity checks for network communication are skipped for job file transfers. This was a choice made years ago to minimize cpu overhead computing the checksums, which probably doesnât make sense today.
We are looking at adding code to do end-to-end integrity checks for file transfers. Todd Tannenbaum would best know the state of that work.