[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Conditionally add default environment variables



What is the cleanest and most robust way for Condor 10.0.x on Linux to add an environment variable assignment to a job's Environment ClassAd attribute at job submission time conditional on there not already being an assignment?

	As far as I know, the only way to do this is with job transforms.

If there isn't a first class job classadd method to conditionally manipulate the Environment attribute, does anyone have a working JOB_TRANSFORM example that has all the right string escaping and error handling?

mergeEnvironment() will do what you want if you pass the default value(s) first.

P.S. More generally, does Condor attempt any checks or control over the plethora of environment variables that can be at variance with a job's request_cpu value, e.g., OMP_NUM_THREADS, MPI_NUM_THREADS, MKL_NUM_THREADS, OPENBLAS_NUM_THREADS, KMP_AFFINITY ... While Condor's cgroup support is very helpful at preventing a job for using more than it's fair share of cpu resources I am wondering about the converse where one or more standard multi-threading libraries has gotten confused and is limiting a job to use less than the allocated request_cpu assignment?

HTCondor sets a number of these variables. The list is in the config variable STARTER_NUM_THREADS_ENV_VARS. HTCondor currently assumes that the user knows better and won't change variables set in the job's environment specification. Note that KMP_AFFINITY appears to be special, because its value is not the number of threads to use.

- ToddM