[HTCondor-users] Conditionally add default environment variables

["Anderson, Stuart B." <sba@xxxxxxxxxxx>]

What is the cleanest and most robust way for Condor 10.0.x on Linux to add an environment variable assignment to a job's Environment ClassAd attribute at job submission time conditional on there not already being an assignment?

The most recent use case for this is to set KMP_AFFINITY="reset" by default for jobs that don't explicitly request something else. It was recently noticed on a LIGO Condor pool that somewhere around llvm-openmp version 15 the default behavior of applications using the Python multiprecessing.Pool fork method changed to unfortunately set the cpu affinity for Vanilla Universe child processes such that the full set of processes in a job are limited to running on fewer than their requested, and Condor allocated, cpu cores.

If there isn't a first class job classadd method to conditionally manipulate the Environment attribute, does anyone have a working JOB_TRANSFORM example that has all the right string escaping and error handling?

P.S. More generally, does Condor attempt any checks or control over the plethora of environment variables that can be at variance with a job's request_cpu value, e.g., OMP_NUM_THREADS, MPI_NUM_THREADS, MKL_NUM_THREADS, OPENBLAS_NUM_THREADS, KMP_AFFINITY ... While Condor's cgroup support is very helpful at preventing a job for using more than it's fair share of cpu resources I am wondering about the converse where one or more standard multi-threading libraries has gotten confused and is limiting a job to use less than the allocated request_cpu assignment?

Thanks.
 
--
Stuart Anderson
sba@xxxxxxxxxxx