Re: [HTCondor-users] File transfers and symlinks along the path

[Steffen Grunewald <steffen.grunewald@xxxxxxxxxx>]

On Mon, 2023-05-01 at 15:06:23 +0000, HTCondor Users Mailinglist wrote:
> I donât know why HTCondor would add a bunch of â../â path elements. It generally leaves pathnames untouched, pastes the Iwd to the front of relative paths, or uses realpath() to get the canonicalized absolute path (again using Iwd for relative paths).
> 
> With jobs that have an Iwd that starts with â/.auto/home/â, what do the file transfer attributes look like (e.g. TransferInput)?

I wish I could have presented such a job not only from memory, but the affected user
decided to remove the held jobs instead of leaving them for closer inspection.
Whatever does the path modifications, it ends up with /.auto/home/........./home/.........
which contains (contained, as we have modified our mount strategy in a hurry) a symlinked
/home/ right in the middle, causing the file transfer to complain.

Since the underlying "problem" (our approach at auto-mounting) has been "resolved"
only the question remains "why" - iow, the rationale behind symlinks being forbidden
- as this may hit us in another place, too.

Thanks,
 Steffen


> 
>  - Jaime
> 
> > On Apr 28, 2023, at 6:00 AM, Steffen Grunewald <Steffen.Grunewald@xxxxxxxxxx> wrote:
> > 
> > Good afternoon,
> > 
> > we've been running our HTC clusters for many years now, always automounting
> > the "home" tree (served by a single NFS server) as a whole into /.auto and
> > symlinking /home -> /.auto/home (as automounting individual homes per-user
> > looked like wasted cycles to us).
> > All the time we didn't face any issues with that approach - in particular
> > because file transfer wasn't necessary at all, with filesystems shared
> > across the whole pool.
> > 
> > 
> > Now the situation has changed, and while we suspect that our job generator
> > is part of the problem and mixing up the results of `pwd` and `pwd -P` (or
> > the equivalent in Python code), we also see HTCondor contributing to it:
> > 
> > We now frequently find transfer file paths that start with a bunch of "../"
> > (to traverse back to the / directory) and then append "home/${user}/foo/bar/baz"
> > to that when the relative path would just be "baz".
> > "initialdir" (in job classAd: Iwd) is set to "/.auto/home/${user}/foo/bar".
> > This doesn't cause any problems anywhere else, it's just the shadow that
> > finds out about the symlink right in the middle of the path and refuses to
> > transfer the file - causing the job to be held.
> > 
> > 
> > My questions now are:
> > Since this is intended behaviour, may I learn about the rationale behind it?
> > Is there a means to work around it in HTCondor, or do we have to abandon
> > the old mount strategy?
> > 
> > 
> > Thanks,
> > Steffen

-- 
Steffen Grunewald, Cluster Administrator
Max Planck Institute for Gravitational Physics (Albert Einstein Institute)
Am MÃhlenberg 1 * D-14476 Potsdam-Golm * Germany
~~~
Fon: +49-331-567 7274
Mail: steffen.grunewald(at)aei.mpg.de
~~~