Re: [HTCondor-users] condor_ssh_to_job broken with 8.8 on CentOS 7

[Oliver Freyermuth <freyermuth@xxxxxxxxxxxxxxxxxx>]

Dear Greg,

Am 26.02.19 um 18:55 schrieb Greg Thain:
> On 2/26/19 11:48 AM, Oliver Freyermuth wrote:
> > Dear Greg,
> >
> > Am 26.02.19 um 18:31 schrieb Oliver Freyermuth:
> > > So probably, this only fails for interactive jobs, since the sleep is reaped before we attach?
> > > I can't test witha batch job right now since I am already in the middle of the downgrade (and we still lack a proper test setup), but I'll try.
> >
> > Indeed that's the case. Replacing "-a" with "-m -p -u -U" via a wrapper (thanks Christoph!) makes attaching to running non-interactive batch jobs via condor_ssh_to_job
> > almost work.
> > However, attaching to the user namespace fails:
> > nsenter: reassociate to namespace 'ns/user' failed: Invalid argument 
>
>
> If your singularity is configured to launch containers with the setuid wrapper (the default), you don't want the -U in the command line options for nsenter.

indeed that's the case for us as of now.
I have also tried without "-U" (but wouldn't -a also include -U, since /proc/<pid>/ns/user exists even for setuid singularity?).
Without -U, I can indeed attach to a running batch job, but I get:

+ /usr/bin/nsenter -m -p -u -t 24166 /usr/sbin/chroot --userspec 67803 /proc/24166/root
sh: cannot set terminal process group (-1): Inappropriate ioctl for device
sh: no job control in this shell
sh-4.1$

Since this seems to fire up a standard "sh" and not the shell Singularity invokes, it seems that also the environment usually
set up by singularity is missing (i.e. even PATH is empty). Of course, that can be fixed by "source /etc/profile" manually,
so I'd say that this shows that things work in general apart from the still rough edges ;-).

Since our users are still used to the environment you get when running sshd inside the container, we have to do the migration more slowly,
and then I would actually wait until the issues with interactive jobs and "nsenter -a" are fixed (without using a wrapper as a hack) before approaching the upgrade.

Cheers and thanks!
	Oliver

> -greg
>
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/


--
Oliver Freyermuth
UniversitÃt Bonn
Physikalisches Institut, Raum 1.047
NuÃallee 12
53115 Bonn
--
Tel.: +49 228 73 2367
Fax:  +49 228 73 7869
--

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature