Re: [HTCondor-users] how to refer to executable not existing in the submitters context?

[Mark Coatsworth <coatsworth@xxxxxxxxxxx>]

Hi Thomas,

A better way to do this for Singularity is to put your executable in a local /scratch directory, which gets automatically mounted into the container. That way your file will exist in the same place both inside the container and your local submitter context. Your submit file would then include the following lines:

executable = /scratch/my-executable.sh

should_transfer_files = YES

...

The scratch mount for Singularity generally works the same way as for Docker universe jobs, which is explained in our manual: https://htcondor.readthedocs.io/en/latest/users-manual/docker-universe-applications.html

Mark

On Wed, May 27, 2020 at 8:10 AM Thomas Hartmann <thomas.hartmann@xxxxxxx> wrote:

Hi all,

I am a bit at a loss here, but how can I refer to an executable, that
does not exists on the submitter's context.

I.e., I have a Singularity container where the executable is available under
  /opt/spark/sbin/start-slave.sh
that exists just in the container's context. The job is submitted with
the Singularity image as target.

Also with 'should_transfer_files = NO', condor_submit fails with [1],
since the executable does not exists under the path at submission.

Christoph suggested a detour via /bin/bash as executable and the actual
script/application as argument [2] - which is a bit hacky, but works ;)

So, I wonder if there is a better(?) way/what am I missing here?

Cheers,
  Thomas


[1]
ERROR: Executable file /opt/spark/sbin/start-slave.sh does not exist


[2]
> test.submit
should_transfer_files = NO
Executable     = /usr/bin/bash
arguments      = "/opt/spark/sbin/start-slave.sh --port 3100 ..."
...
Queue 1
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

--

Mark Coatsworth

Systems Programmer

Center for High Throughput Computing

Department of Computer Sciences

University of Wisconsin-Madison