Re: [HTCondor-users] Propagate DOCKER_CONFIG to docker universe

["Dr. Markus Stabrin" <Markus.Stabrin@xxxxxxxxxxxxxxxxxxx>]

Hello everybody,

actually while writing my previous mail I had an idea about how to solve it and apparently it works, so I will share my finding:

I use the PreCmd to pull the image into the local cache using the DOCKER_CONFIG variable.

The docker universe then uses the pre-fetched image to execute the job.

The PostCmd then deletes the image.

I think this could lead to some issues when the image gets removed while being used by other jobs, so something more sophisticated might be more suitable, but it works for now:

```
universe                = docker
docker_image            = <private-image>
RunAsOwner              = True
output                  = stdout.log
error                   = stderr.log
log                     = stdlog.log

+PreCmd = "/usr/bin/docker"
+PreArguments = "pull <private-image>"
+PreEnvironment = "DOCKER_CONFIG=/etc/docker/userconf"
+PostCmd = "/usr/bin/docker"
+PostArguments = "image rm -f <private-image>"
queue

```

If there are other best-practice solutions, I am still open for suggestions ð

Best,

Markus

---

From: Dr. Markus Stabrin
Sent: Thursday, May 11, 2023 1:35:00 PM
To: htcondor-users@xxxxxxxxxxx
Subject: Propagate DOCKER_CONFIG to docker universe

 

Hello everybody,

I am new to Condor and I am struggling to connect to a private docker registry, which I have no control over, so a nginx proxy does not work.

I also checked this: https://www-auth.cs.wisc.edu/lists/htcondor-users/2017-October/msg00113.shtml

But unfortunately it does not seem to work for me and I get an `access forbidden` and I do not find much more information about the topic.

I am running a setup where the actual owner is executing the job and the credentials are stored in `${HOME}/.docker/config.json`.

This fails:
```
universe                = docker
docker_image            = <private-image>
RunAsOwner              = True
output                  = stdout.log
error                   = stderr.log
log                     = stdlog.log
queue
```

But this works:
```
executable              = /usr/bin/docker
arguments               = "pull <private-image>"
output                  = stdout.log
error                   = stderr.log
log                     = stdlog.log
RunAsOwner              = True
queue
```

as well as if there is no home directory at all and the credentials are stored in `/etc/docker/userconf/config.json`:

```

executable              = /usr/bin/docker
arguments               = "pull <private-image>"
output                  = stdout.log
error                   = stderr.log
log                     = stdlog.log
RunAsOwner              = True

environment             = "DOCKER_CONFIG=/etc/docker/userconf"
queue
```

So apparently, the docker universe does not read the `config.json` for the user or from the DOCKER_CONFIG variable?

I tried to put
```

DOCKER_CONFIG = /etc/docker/userconf
```

in `/etc/condor/config.d/01-execute.config` , but this does not seem to be applied.

I also tried to set the variable in a PreEnvironment, but I read that it is not propagated to the execution environment.

Does anyone have further ideas how to use the docker universe with a private docker registry?

Thank you very much,

Markus