Setting OMP_NUM_THREADS to the number of CPUs requested is appropriate behavior in the domain in which HTCondor operates:
Per this spec, the environment variable sets the default number of threads to be used in a parallel region.
If things are forking other things and creating new parallel regions inside other parallel regions, then it seems like it would be incumbent upon the highest level of the application which was launched by HTCondor to manage how many threads child processes get, either by an explicit omp_set_num_threads() call or setting the environment variable after the fork but before the exec.
For example, if youâre running MATLAB, the maxNumCompThreads(1) call is a handy one to have, perhaps in an âif deployedâ statement in the code so it will use all available CPU on a workstation while being run by a user but only one thread when running compiled. I ran into this kind of problem very hard a year or two ago, where 64 MATLAB processes were each spawning 64 threads on a 64-core machine.