Re: [HTCondor-users] set_DockerImage for local image

[Todd Tannenbaum <tannenba@xxxxxxxxxxx>]

On 5/28/2021 3:21 AM, jcaballero.hep@xxxxxxxxx wrote:

> Hi,
>
> I understand that the image set in classad DockerImage is being pulled
> from DockerHub, is that correct?
> I would be interested in using the image from local filesystem rather
> than a docker registry.
> Is that something that can be done via that, or any other, classad?
>
> Thanks a lot in advance.
> Cheers,
> Jose
>     
    
    Hi Jose,
    
    IIRC, the image set in the DockerImage is currently only pulled over
    the network from a hub if the image is not already stored on local
    disk in Docker's own image cache (typically in /var/lib/Docker/...).
    Not sure if this fits your use case, but if you just want some
    special image to be available on all your worker nodes, as the
    sysadmin you could issue a "docker load ..." or "docker import ..."
    command (depending on if you created the image on your local
    filesystem with 'docker save' or 'docker export', respectively) on
    each worker node outside of HTCondor in order to copy the image from
    local disk into Docker's own local image cache.    Then jobs could
    use this image via Docker Universe as usual, no hub involved.   
    
    One complication: by default HTCondor does garbage collection of
    images in the Docker local cache to keep the number if images there
    from infinitely growing.  If your Docker Universe jobs use a many
    different images, this could result in the special image you loaded
    into the cache being removed.   You may want to customize HTCondor
    config knob DOCKER_IMAGE_CACHE_SIZE to prevent this from happening.
    
    Finally, more flexible docker image management is something we are
    planning to improve in HTCondor v9.1.x series.  Including support
    for private images, and images transferred by HTCondor file transfer
    as part of the job's usual sandbox of files.  So hopefully the above
    will get easier in the near future.
    
    Hope this helps,
    
    regards,
    Todd
    
    
    
    
    
    
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/
>     
    
    
-- 
Todd Tannenbaum <tannenba@xxxxxxxxxxx>  University of Wisconsin-Madison
Center for High Throughput Computing    Department of Computer Sciences
Calendar: https://tinyurl.com/yd55mtgd  1210 W. Dayton St. Rm #4257
Phone: (608) 263-7132                   Madison, WI 53706-1685