[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Docker force ignore cache



On 10/5/2016 3:35 PM, Michael Fienen wrote:
Hi HTCondorians

I am running in docker universe. The image I created required a very
small change. However, I had already run a job with that image, so it
was in the cache on most of my workers. If I delete it manually from
the cache join a single machine and run it again, it downloads the
new updated version. Is there a way to do this via htcondor? Would be
nice to instruct HTCondor that I need a fresh pull of the image.


I am not a docker universe expert, but I think that HTCondor's docker universe does the equal of a "docker run". Perhaps it should be doing the equal of a docker pull followed by a docker run; seems like that would be most intuitive, but of course the bad news would be more hits to your hub.

Something I think you could do right now to deal with the above is I believe the docker image you specify in your submit file can be decorated to force a specific image version, by either tag name or digest. If the version in the cache is not what you specified in your submit file, it will not be used, and the correct version should be fetched and cached. You do this by just appending ":tagID" or "@digestID" to the name of the image in your submit file.

Some examples (shamelessly copied from the docker documentation) appear below.

Hope this helps
Todd

Image[:tag]

While not strictly a means of identifying a container, you can specify a version of an image you’d like to run the container with by adding image[:tag] to the command. For example, docker run ubuntu:14.04.

Image[@digest]

Images using the v2 or later image format have a content-addressable identifier called a digest. As long as the input used to generate the image is unchanged, the digest value is predictable and referenceable.

The following example runs a container from the alpine image with the sha256:9cacb71397b640eca97488cf08582ae4e4068513101088e9f96c9814bfda95e0 digest:

$ docker run alpine@sha256:9cacb71397b640eca97488cf08582ae4e4068513101088e9f96c9814bfda95e0 date



Thanks! Mike Fienen USGS Wisconsin Water Science Center
_______________________________________________ HTCondor-users
mailing list To unsubscribe, send a message to
htcondor-users-request@xxxxxxxxxxx with a subject: Unsubscribe You
can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/



--
Todd Tannenbaum <tannenba@xxxxxxxxxxx> University of Wisconsin-Madison
Center for High Throughput Computing   Department of Computer Sciences
HTCondor Technical Lead                1210 W. Dayton St. Rm #4257
Phone: (608) 263-7132                  Madison, WI 53706-1685