Re: [HTCondor-users] howto set ulimit -l unlimited for openmpiscript or other condor jobs on debian 9 for htcondor 8.8?

[Tim Theisen <tim@xxxxxxxxxxx>]

Hello Harald,

Sorry, that was my fault. I used 'infinity' rather than 'unlimited' in
my test VM. I couldn't cut and paste easily, so I just typed it wrong.

...Tim

On 3/29/19 4:14 PM, Harald van Pee wrote:
> Hello Tim,
>
> thank you for your good explanation in principle this works:
> condor.service - Condor Distributed High-Throughput-Computing
>    Loaded: loaded (/lib/systemd/system/condor.service; enabled; vendor preset: 
> enabled)
>   Drop-In: /etc/systemd/system/condor.service.d
>            ââcondor.conf
> BUT:
> LimitMEMLOCK=unlimited
>
> results still in 
> max locked memory       (kbytes, -l) 64
>
> therefore I tried numbers and found out 
> LimitMEMLOCK=32768
> results in 
> max locked memory       (kbytes, -l) 32
>
> LimitMEMLOCK=99999999
> results in 
> max locked memory       (kbytes, -l) 97656
>
> LimitMEMLOCK=9999999999
> results in 
> max locked memory       (kbytes, -l) 9765624
>
> obviously something goes wrong but at least it seems high numbers can be used 
> as a work around.
>
> Now the question is is the problem in htcondor or systemd or the debian 
> kernel? Or do I still something wrong?
>
> Best
> Harald
>
>
> On Friday, March 29, 2019 3:49:21 PM CET Tim Theisen wrote:
>> Hello Harald,
>>
>> The /etc/init.d/condor file is not used. Distributing it was a mistake
>> on our part.
>>
>> So, we should modify the systemd configuration.
>>
>> When updating the condor service, one should not modify the system
>> installed service file. Instead, create a file with overrides to the
>> distributed configuration. Here are the steps:
>>
>> 1. mkdir /etc/systemd/system/condor.service.d 2. Put the following 2 lines
>> into
>  /etc/systemd/system/condor.service.d/condor.conf
>> [Service]
>> LimitMEMLOCK=unlimited
>>
>> 3. Force a systemd reload: systemctl daemon-reload
>>
>> 4. Finally, restart HTCondor: systemctl restart condor
>>
>> ...Tim
>>
>> On 3/28/19 3:56 PM, Harald van Pee wrote:
>>
>>> Dear htcondor experts,
>>>
>>>
>>>
>>> we running htcondor 8.8.1 on debian 9 with vanilla universe jobs without 
>>> problems and now want to start openmpi jobs in parallel universe, but
>>> to do so 
>>> we need to set max locked memory to a high value
>>> I have set
>>> ulimit -l unlimited
>>> in 
>>> /etc/init.d/condor
>>> and
>>> the openmpiscript itself.
>>> And added to
>>> /lib/systemd/system/condor.service
>>> [Service]
>>> ...
>>> LimitMEMLOCK=unlimited
>>>
>>>
>>>
>>>
>>> then I have down a
>>> systemctl daemon-reload
>>> systemctl restart condor
>>>
>>>
>>>
>>> on all condor hosts.
>>>
>>>
>>>
>>> But after starting openmpiscript 
>>> ulimit -l 
>>> inside of the script shows
>>> max locked memory       (kbytes, -l) 64
>>>
>>>
>>>
>>> And as expected the job does not run properly.
>>>
>>>
>>>
>>> What I have done wrong? How I have to set the
>>> max locked memory
>>> limit for condor jobs/scripts?
>>>
>>>
>>>
>>> Best regards
>>> Harald
>>> _______________________________________________
>>> HTCondor-users mailing list
>>> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with
>>> a
>  subject: Unsubscribe
>>> You can also unsubscribe by visiting
>>> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>>>
>>>
>>>
>>> The archives can be found at:
>>> https://lists.cs.wisc.edu/archive/htcondor-users/
>>
>> -- 
>> Tim Theisen
>> Release Manager
>> HTCondor & Open Science Grid
>> Center for High Throughput Computing
>> Department of Computer Sciences
>> University of Wisconsin - Madison
>> 4261 Computer Sciences and Statistics
>> 1210 W Dayton St
>> Madison, WI 53706-1685
>> +1 608 265 5736
>>
>>
>
-- 
Tim Theisen
Release Manager
HTCondor & Open Science Grid
Center for High Throughput Computing
Department of Computer Sciences
University of Wisconsin - Madison
4261 Computer Sciences and Statistics
1210 W Dayton St
Madison, WI 53706-1685
+1 608 265 5736