Re: [HTCondor-users] Executable is not overriding the ENTRYPOINT In docker universe

[Andres Ayala <Andres.Ayala@xxxxxxxxxxxx>]

Hi Greg,

I was afraid that the only option was to change the docker images because
In our case this might be difficult (users are providing the images, many times with the precise sha256 hash, and they use the entrypoint for more "human friendly" actions)

At the beginning I found a workaround that might help others.
Putting in the executors config files:

DOCKER_EXTRA_ARGUMENTS = --entrypoint=

Reset the entrypoint and force docker to use as executable the first argument.

Unfortunately we need the behaviour to work per-image, as sometimes the entry point is really the interpreter (for example, the right python to use)

I looked a little bit in condor sources and found they are easy to understand and build (the build containers are nice),
so I managed to create a modified version with a job command to control how the entrypoint is used.

The command flag uses as default "false" to obtain the current behaviour, but including in the submit:
docker_override_entrypoint = true

with a non-empty executable, the docker create call build in src/condor_utils/docker-api.cpp will look like:
        docker create ... --entrypoint executable docker_image args
Instead of the default:
        docker create ... docker_image executable args

I do not know if this might be interesting to other people (possibly we are the only ones with this necessity)
I can prepare the patch/pull request, with the tests needed to check this functionality,
but I perfectly understand if you do not find it useful or usually do not accept change proposals.

Thanks for your support!
AndrÃs

EUMETSAT
AndrÃs Ayala
Data Processing Systems Engineering


-----Original Message-----
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Greg Thain via HTCondor-users
Sent: Saturday, February 10, 2024 6:20 AM
To: htcondor-users@xxxxxxxxxxx
Cc: Greg Thain <gthain@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] Executable is not overriding the ENTRYPOINT In docker universe

WARNING: This email originated outside the organisation. Treat links and attachments with caution.


On 2/6/24 8:38 AM, Andres Ayala wrote:
> Hi all,
>
> I am trying to launch several jobs in the docker universe.
>
> The image is the same for all of them, but it contains many binaries that can be executed independently.
> The image has also defined an ENTRYPOINT to other binary.

Hi Andres:

Thanks for your report.  Sharing a common docker image, and using it to run several different jobs (by using a different executable) is a common use case that we see a lot.  Generally speaking, though, the intent of ENTRYPOINT in a docker image is that it isn't often overridden.  Usually it is something like a shell or other program launcher,  that can run the CMD that is either built into the image, or overridden on the command line.

Usually docker images that want to run some default program when started make that the program the CMD rather than the ENTRYPOINT. I'm afraid we don't have a good workaround in HTCondor for this. Could you rebuild this container, and set the ENTRYPOINT to "/bin/bash"?  I think that will solve the problem.


-greg


>
> I am trying to create each job with a job like:
>
> universe     = docker
> docker_image = my_image:tag
> executable   = /opt/bin/cmd_a
> # The executable present in the image
> transfer_executable   = False,
> arguments             = params.yml
> should_transfer_files = True
> transfer_input_files  = params.yml
> transfer_output_files = result.txt
> # Logs
> output = mode_a.out
> error  = mode_a.err
> log    = mode_a.log
>
> Where /opt/bin/cmd_a changes in each job.
>
> Looking at the condor logs, the docker create command generated is something like:
> Attempting to run: /usr/bin/docker create [may arguments ommited]
>                     --init [tons of -e and --volume and --workdir --user --group]
>                                     my_image:tag /opt/bin/cmd_a
> params.yml
>
>  From the documentation of docker run/create
> https://urldefense.com/v3/__https://docs.docker.com/engine/reference/r
> un/*commands-and-arguments__;Iw!!Mak6IKo!O1CiPQXCXNvkdU2AIVEWJ1stdiyz6
> bC3WCTCVn64SFQJSLkmjYRi8xjrsOv6S5ClEdeQXDiz1Gl6Jir1Q4rdULnhe08$
>
> Seems like this shall work, and docker will run:
> /opt/bin/cmd_a params.yml
>
> The problem is that when condor starts the image the effective command line is:
> /usr/bin/entrypoint /opt/bin/cmd_a params.yml
>
> And this fails.
>
> Looking even more in the docs, in:
> https://urldefense.com/v3/__https://docs.docker.com/engine/reference/b
> uilder/*entrypoint__;Iw!!Mak6IKo!O1CiPQXCXNvkdU2AIVEWJ1stdiyz6bC3WCTCV
> n64SFQJSLkmjYRi8xjrsOv6S5ClEdeQXDiz1Gl6Jir1Q4rdhq3m7pU$
>
> seems that when an entrypoint is defined, docker uses all the arguments of "docker create" for the entrypoint and do not use the first one as command.
>
> The docker create command that will do what I am expecting would look like:
> Attempting to run: /usr/bin/docker create [may arguments ommited]
>                     --init [tons of -e and --volume and --workdir --user --group]
>                                     --entrypoint /opt/bin/cmd_a
> my_image:tag params.yml
>
> Is there any option for the docker job that modifies the entrypoint to obtain something like this?
>
> Thanks!
> AndrÃs Ayala
>
> Any email message from EUMETSAT is sent in good faith but shall neither be binding nor construed as constituting a commitment by EUMETSAT, except where provided for in a written agreement or contract or if explicitly stated in the email. Please note that any views or opinions presented in this email are solely those of the sender and do not necessarily represent those of EUMETSAT. This message and any attachments are intended for the sole use of the addressee(s) and may contain confidential and privileged information. Any unauthorised use, disclosure, dissemination or distribution (in whole or in part) of its contents is not permitted. If you received this message in error, please notify the sender and delete it from your system.
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx
> with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

Any email message from EUMETSAT is sent in good faith but shall neither be binding nor construed as constituting a commitment by EUMETSAT, except where provided for in a written agreement or contract or if explicitly stated in the email. Please note that any views or opinions presented in this email are solely those of the sender and do not necessarily represent those of EUMETSAT. This message and any attachments are intended for the sole use of the addressee(s) and may contain confidential and privileged information. Any unauthorised use, disclosure, dissemination or distribution (in whole or in part) of its contents is not permitted. If you received this message in error, please notify the sender and delete it from your system.