I’ve built a Singularity definition file that installs Miniconda and creates an environment YAML file, then builds the environment, and then configures the container
so that the environment is activated automatically at the startup of the Singularity container. With Miniconda a CUDA 10.2 and 18.04 Ubuntu Singularity container file is about 850 megabytes in size.
Singularity doesn’t necessarily have an extra infrastructure layer, as it doesn’t require any services from the host – I bet it would be possible to input-transfer
the Singularity executable and run it on an input-transferred container.
Alternatively, you could build out the full virtualenv in a directory with Miniconda, and then input-transfer that whole directory and activate it when the job
starts up, which would eliminate the need for modules to be available on the exec node.
Michael V Pelletier
Digital Transormation & Innovation
I am trying to run an analysis on my local workstation pool that relies on software in a conda virtual environment. When the job runs on a remote machine, it does not have access to the libraries
in that env back on the submit machine.