Re: [HTCondor-users] Singularity and Charliecloud support for HTCondor?

[Oliver Freyermuth <freyermuth@xxxxxxxxxxxxxxxxxx>]

> Am 30.10.2017 um 16:13 schrieb Greg Thain:
>> On 10/29/2017 02:29 PM, Oliver Freyermuth wrote:
>>> However, it has several basic flaws related to interactive jobs / condor_ssh_to_job:
>>> - condor_ssh_to_job ends up in it's own namespaces instead of those of the job
>>
>> This is something we are actively thinking about, and would like to fix, both for singularity and docker universe. The basic problem is that we'd really like to run the sshd as non-root, and outside of the container. However, nsenter requires root to enter a container.
> 
Dear Greg,

researching a bit, I think the solution for Docker could be:
- docker exec (which likely uses support from the Daemon running as root internally)

For Singularity's privileged containers, you can change the approach since 2.4.
Use:
- "singularity instance.start" instead of "singularity exec"
- Then use "singularity exec instance://instancename executable" to run the job. 
- "singularity exec instance://instancename " then allows you to run any binary in the job's environment. 
Of course, this relies on Singularity being setuid root, which is the default. 

For this reason, you could call these commands from the sshd running unprivileged on the host,
using the (already used) command injection approach (ForceCommand or via AuthorizedKeys). 

For singularity's unprivileged containers, which sysadmins will surely prefer if possible,
unprivileged nsenter should also work after fixing an issue in singularity (it already does, but you enter into the system's root, not the container's root, with user privileges). 
Since Singularity currently advertises this issue as a feature, I am not sure they can be convinced to fix it, though. 

For Charliecloud (which runs unprivileged by default), things work out of the box with unprivileged nsenter. 

Please let me know:
- whether there are issues with these ideas which I do not see yet. 
- whether charliecloud (which offers some significant security benefits since it operates fully unprivileged) is also on your list to integrate. 
  I'd strongly prefer to see this as cluster admin, since it allows me to offer the same features to our users without
  any root-running daemon (Docker, which however has at least has seen some security testing) or setuid root code (Singularity, which has still pretty young code)
  running on our machines. It can run Docker containers and also extracted singularity containers without modification. 
- whether it would help if I helped out, for example to write something for Charliecloud integration, close to the existing Singularity integration in HTCondor. 
- whether I'd better redirect these questions to htcondor-devel. 

Cheers and many thanks in advance,
Oliver

> 
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/
>

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature