Re: [HTCondor-users] Limiting HTCondor total RAM usage

[Tim St Clair <tstclair@xxxxxxxxxx>]

FWIW - this is easily done via systemd slices, you can add the block below to your unit file. 

The memory limit will bound the unit.

#######################################
# NOTE: Below are cgroup options
#######################################
Slice=condor
CPUAccounting=true
#######################################
# NOTE: Adjust as needed
#######################################
CPUShares=1024
 
##################
MemoryAccounting=true
#TBD: MemoryLimit=bytes, MemorySoftLimit=bytes
##################
#BlockIOAccounting=true
#BlockIOWeight=??
#BlockIODeviceWeight=??
#TBD: BlockIOReadBandwidth=bytes, BlockIOWriteBandwidth=bytes
 
Cheers,
Tim

----- Original Message -----
> From: "Brian Candler" <b.candler@xxxxxxxxx>
> To: "HTCondor-Users Mail List" <htcondor-users@xxxxxxxxxxx>
> Sent: Tuesday, February 17, 2015 6:47:19 AM
> Subject: [HTCondor-users] Limiting HTCondor total RAM usage
> 
> Does anyone have experience in hard-limiting condor's *total* RAM usage,
> e.g. by putting the condor_start process inside a cgroup?
> 
> I have some machines which need to share with some critical background
> tasks, and I need to avoid them being hit by the OOM killer (which can
> happen if I throw a load of jobs into the queue, and those jobs have
> underestimated their RAM usage)
> 
> I found this:
> https://htcondor-wiki.cs.wisc.edu/index.cgi/wiki?p=HowToLimitMemoryUsage
> 
> But if I understand it rightly, this is for enforcing a hard memory
> usage on individual jobs. This might work, but (a) it involves all the
> RequestMemory values being individually correct, and (b) I have a lot of
> jobs which share memory, e.g. they mmap() the same file, and I'm not
> sure how a hard RequestMemory limit would interact with that.
> 
> So I'd rather for now limit the total HTCondor usage only. Putting
> HTCondor inside a VM is one option (messy); deploying a HTCondor Docker
> container is another option (more stuff to learn); so I wonder if using
> a cgroup directly might be the way to go.
> 
> Any experiences greatfully received.
> 
> Regards,
> 
> Brian Candler.
> 
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/
> 

-- 
Cheers,
Timothy St. Clair
Red Hat Inc.