[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] OMP_NUM_THREADS forced to request_cpus value





On Thu, Mar 15, 2018 at 12:49 AM, Michael Pelletier <Michael.V.Pelletier@xxxxxxxxxxxx> wrote:

Setting OMP_NUM_THREADS to the number of CPUs requested is appropriate behavior in the domain in which HTCondor operates:Â

Â

https://msdn.microsoft.com/en-us/library/xh7e1zba.aspx

https://msdn.microsoft.com/en-us/library/bx15e8hb.aspx

Â

Per this spec, the environment variable sets the default number of threads to be used in a parallel region.



As a general matter, I think I'd disagree. Not so much on grounds that it does't make sense for some applications, but rather it seems an odd way to ensure that an application doesn't exceed its resource allotment. If that were the goal, I would think something like taskset would be more appropriate.ÂOpenMP is inherently about multi-threading within a process, not really about total thread usage of an entire job, which may be a much more complex structure, so I don't think in general one can assume that relation.

Of course, one can re-design an application around this behavior of condor, and in the short term we'll do so. Here,ÂI think condor is assuming a little too much about the behavior of jobs to this environment variable.ÂIn any event, if it maybe become possible to set this variable in the future, I think that would be very useful.
Â

Â

If things are forking other things and creating new parallel regions inside other parallel regions, then it seems like it would be incumbent upon the highest level of the application which was launched by HTCondor to manage how many threads child processes get, either by an explicit omp_set_num_threads() call or setting the environment variable after the fork but before the exec.Â

Â

For example, if youâre running MATLAB, the maxNumCompThreads(1) call is a handy one to have, perhaps in an âif deployedâ statement in the code so it will use all available CPU on a workstation while being run by a user but only one thread when running compiled. I ran into this kind of problem very hard a year or two ago, where 64 MATLAB processes were each spawning 64 threads on a 64-core machine.

Â

ÂÂÂÂÂÂÂÂÂÂÂ -Michael Pelletier.


_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@cs.wisc.edu with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/