[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Docker in Condor



Hi Luke,

I'm interested in using Docker for HTCondor job executables also but have similarly come up empty on examples for doing that (although it has been mentioned on this list a couple times). The tricky bit has to do with the docker executable needing sudo and Condor's policy against root permissions for user code and so we probably need a small patch to make that work out or perhaps just an expert sysadmin who knows how to set it up.

In the meantime I've gone the other direction and put Condor in a Docker container orchestrated by Kubernetes. Most of the benefits of dockerization are available (except for container composition which is an important capability with Docker) with the use case being ephemeral clusters using cloud provisioning. Â

My new blog has a relevant post at:Âhttp://jimwhite.github.io/Â. The example doesn't yet include a submit minion to show how a whole workflow can be run. I hope to get back to this project in a few weeks with a more complete demo including (hopefully) auto-scaling.

Jim

On Fri, Dec 5, 2014 at 4:27 PM, Luke Winslow <lwinslow@xxxxxxxx> wrote:
Hi All,

Quick question. Has anyone experimented with using Docker as a job container internal to Condor? We are experimenting with it to improve how we manage the job environment for Condor jobs. Just curious if anyone was doing anything already. General searches (as well as here) didn't point me to any examples.

Thanks!

-Luke

--
Center for Integrated Data Analytics
USGS Wisconsin Water Science Center
8505 Research Way
Middleton, WI 53562
Office: 608-821-3914
Cell: 608-332-1245

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@cs.wisc.edu with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/