[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Singularity not mounting the home directory for a submitted job


What I'd wager is happening is that you have

mount home = yes

in your singularity.conf file. This has the effect of bind mounting $HOME onto /home/user-running-singularity in the container. Which means you can't bind mount host:/home onto the non-empty container:/home. If you have a situation where a job needs BOTH /home/A and /home/B, you should probably set "mount home = no" and then 

bind path = /home

or the corresponding  SINGULARITY_BIND_EXPR.


ïOn 10/15/18, 4:50 PM, "HTCondor-users on behalf of Todd Tannenbaum" <htcondor-users-bounces@xxxxxxxxxxx on behalf of tannenba@xxxxxxxxxxx> wrote:

    On 10/15/2018 1:42 PM, Michael Pelletier wrote:
    > Hi folks,
    > Does anyone have some insight into how Singularity decides when to bind-mount the user's home directory?
    > I've got a situation where when I run the command on the exec machine directly, the container spins up successfully with the home directory in place, but when started via condor_run -a 'MY.SingularityImage = "imagefile"' or via a submit description, the container doesn't get the home directory and thus the launch of the application fails.
    > The bind mount arguments include the /home/condor directory since that's where the global libexec lives in my config, but omitting that didn't change the behavior. An error message I see is "/home/user already mounted in container" - so it's apparently detecting something that isn't actually there.
    > I have a user_job_wrapper in place, but omitting that didn't change this behavior. I wrapped the Singularity binary as well in order to add the "--nv" option for jobs where it's appropriate until SINGULARITY_EXTRA_ARGUMENTS is in a production release, but that was another dead end investigation. I tried inserting a "-H ${HOME}:$HOME" argument along with --nv, but that seems to be getting ignored with the "already mounted in container" message.
    > What's even more peculiar is that it works for some user accounts, but not others. I checked the environment variables for any differences, but nothing jumped out at me, and condor_run does a getenv=True in any case.
    > Any suggestions? Thanks
    I am not the Singularity support expert, but maybe HTCondor config knob 
    SINGULARITY_BIND_EXPR could help?  Ie something like:
       SINGULARITY_BIND_EXPR = strcat("/home/",Owner)
    HTCondor-users mailing list
    To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
    subject: Unsubscribe
    You can also unsubscribe by visiting
    The archives can be found at: