[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] set_DockerImage for local image



hi Todd,

that was very clarifying indeed.
So I guess I will have to modify a little bit the wrapper pointed by
config var DOCKER to "load" the image before running "docker create".
BTW, I intend to store the images in CVMFS, so no /var/lib/Docker
involved here.

Thanks a lot, Todd.
Cheers,
Jose


El vie, 28 may 2021 a las 19:27, Todd Tannenbaum
(<tannenba@xxxxxxxxxxx>) escribiÃ:
>
> On 5/28/2021 3:21 AM, jcaballero.hep@xxxxxxxxx wrote:
>
> Hi,
>
> I understand that the image set in classad DockerImage is being pulled
> from DockerHub, is that correct?
> I would be interested in using the image from local filesystem rather
> than a docker registry.
> Is that something that can be done via that, or any other, classad?
>
> Thanks a lot in advance.
> Cheers,
> Jose
>
>
> Hi Jose,
>
> IIRC, the image set in the DockerImage is currently only pulled over the network from a hub if the image is not already stored on local disk in Docker's own image cache (typically in /var/lib/Docker/...). Not sure if this fits your use case, but if you just want some special image to be available on all your worker nodes, as the sysadmin you could issue a "docker load ..." or "docker import ..." command (depending on if you created the image on your local filesystem with 'docker save' or 'docker export', respectively) on each worker node outside of HTCondor in order to copy the image from local disk into Docker's own local image cache.    Then jobs could use this image via Docker Universe as usual, no hub involved.
>
> One complication: by default HTCondor does garbage collection of images in the Docker local cache to keep the number if images there from infinitely growing.  If your Docker Universe jobs use a many different images, this could result in the special image you loaded into the cache being removed.   You may want to customize HTCondor config knob DOCKER_IMAGE_CACHE_SIZE to prevent this from happening.
>
> Finally, more flexible docker image management is something we are planning to improve in HTCondor v9.1.x series.  Including support for private images, and images transferred by HTCondor file transfer as part of the job's usual sandbox of files.  So hopefully the above will get easier in the near future.
>
> Hope this helps,
>
> regards,
> Todd
>
>
>
>
>
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/
>
>
>
> --
> Todd Tannenbaum <tannenba@xxxxxxxxxxx>  University of Wisconsin-Madison
> Center for High Throughput Computing    Department of Computer Sciences
> Calendar: https://tinyurl.com/yd55mtgd  1210 W. Dayton St. Rm #4257
> Phone: (608) 263-7132                   Madison, WI 53706-1685