I am studying the HTCondor Admin Manual to understand how to set up a cycle-scavenging pool here in Exeter. Since the execute machines are Linux workstations that others have root access to, I wasn’t sure how comfortable folks would be having “password-less Docker sudo permission to start the container as root.” It seems that running Docker root-less means forfeiting a bunch of the security protocols the engine uses to keep stuff properly contained.
We haven't tried running HTCondor's docker universe with a rootless docker, but it is certainly something that it on our list. In general, when running with a rootly docker, HTCondor translated attributes of the submit file into a constrained set of docker run commands, so that the job shouldn't be able to get to root or get outside the container. We go one step further than the default docker run command, and right now, at least, HTCondor will never run a job inside the container with the root uid (0).