[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[HTCondor-users] Inconsistent JDL for docker vs vanilla universe



Hi HTCondor developers, all,

Iâve recently experimented with adding Docker in our HTCondor cluster. Technically, it all worked easily and flawlessly.

However, there is one small detail in submitting jobs, which has led to some massive frustration:
JDLs for Docker and Vanilla are fully compatible, but not in how the executable is defined.
Using docker requires our users to jump through hoops in a way that is IMO completely unnecessary, because for Vanilla it is handled by HTCondor.

Does anyone have similar experiences and possible best practices?

Will the Docker universe be fixed/updated to accept vanilla style JDLs?
Would it be possible to use one JDL for *both* vanilla and docker universe?

Some background/details:

- For the vanilla universe, users define their executable as for running it locally. Only files used implicitly must be transferred explicitly:
executable = /path/to/my_app
transfer_input_files = /path/to/my_config

- For docker universe, users must define the executable as being in the current directory. The executable must also be transferred explicitly:
executable = ./my_app
transfer_input_files = /path/to/my_app,/path/to/my_config

- All other universe-dependent settings are simply ignored if not applicable, e.g.:
docker_image = images/my_image  # required by docker, silently ignored by vanilla

I know itâs a very tiny difference, but that makes having to work around it even more frustrating. ;/

Cheers,
Max