[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] how to refer to executable not existing in the submitters context?

Hi Thomas,

A better way to do this for Singularity is to put your executable in a local /scratch directory, which gets automatically mounted into the container. That way your file will exist in the same place both inside the container and your local submitter context. Your submit file would then include the following lines:

executable = /scratch/my-executable.sh
should_transfer_files = YES

The scratch mount for Singularity generally works the same way as for Docker universe jobs, which is explained in our manual: https://htcondor.readthedocs.io/en/latest/users-manual/docker-universe-applications.html


On Wed, May 27, 2020 at 8:10 AM Thomas Hartmann <thomas.hartmann@xxxxxxx> wrote:
Hi all,

I am a bit at a loss here, but how can I refer to an executable, that
does not exists on the submitter's context.

I.e., I have a Singularity container where the executable is available under
that exists just in the container's context. The job is submitted with
the Singularity image as target.

Also with 'should_transfer_files = NO', condor_submit fails with [1],
since the executable does not exists under the path at submission.

Christoph suggested a detour via /bin/bash as executable and the actual
script/application as argument [2] - which is a bit hacky, but works ;)

So, I wonder if there is a better(?) way/what am I missing here?


ERROR: Executable file /opt/spark/sbin/start-slave.sh does not exist

> test.submit
should_transfer_files = NO
Executable     = /usr/bin/bash
arguments      = "/opt/spark/sbin/start-slave.sh --port 3100 ..."
Queue 1
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting

The archives can be found at:

Mark Coatsworth
Systems Programmer
Center for High Throughput Computing
Department of Computer Sciences
University of Wisconsin-Madison