Re: [HTCondor-users] Mixing cgroups, partitionable slots, and CPU limits

[Todd Tannenbaum <tannenba@xxxxxxxxxxx>]

Hi Lincoln,

Just did a quick glance below, but I see two things wrong:

1. As you guessed - do not enable CPU affinity if you are using cgroups.

2. Should you decide to use CPU affinity instead of cgroups, you want to do
   ASSIGN_CPU_AFFINITY = True
and not mess with knob ENFORCE_CPU_AFFINITY - the latter is an expert 
knob that should only be used for special situations (like glideins) 
where there is more than one startd per machine.

best regards
Todd

On 7/13/2015 1:33 PM, Lincoln Bryant wrote:
> Greetings experts,
>
> I have a configuration where Iâm mixing cgroups, partitionable slots, CPU affinity, and limiting the number of CPUs available to HTCondor on my workers. However, it seems like my jobs are all being limited to run on a single core.
>
> I havenât dug into it too deeply, but when the machine is full, âhtopâ shows most of the cores idle and one CPU at 100%. AFAIK, these jobs are very CPU-intensive on other machines.
>
> Is there something obviously wrong in my STARTD config (attached below)?
>
> (Should I even bother with CPU affinity if Iâm using cgroups? )
>
> Cheers,
> Lincoln
>
> NUM_CPUS = 32
> ENFORCE_CPU_AFFINITY = true
>
> SLOT_TYPE_1 = 100%
> SLOT_TYPE_1_PARTITIONABLE = TRUE
> NUM_SLOTS = 1
> NUM_SLOTS_TYPE_1 = 1
> SlotWeight = Cpus
>
> BASE_CGROUP = htcondor
> CGROUP_MEMORY_LIMIT_POLICY = soft
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/


--
Todd Tannenbaum <tannenba@xxxxxxxxxxx> University of Wisconsin-Madison
Center for High Throughput Computing   Department of Computer Sciences
HTCondor Technical Lead                1210 W. Dayton St. Rm #4257
Phone: (608) 263-7132                  Madison, WI 53706-1685