Re: [HTCondor-users] CondorCE: job transform for normalizing jobs' core/mem ratio?

[Thomas Hartmann <thomas.hartmann@xxxxxxx>]

Hi Brian,

yes, from the technical view you are absolutely right.

My worries just go into the 'political direction' ;)

So far, if a VO want's to run highmem jobs, i.e., core/mem < 1/2GB, they
have to scale by cores.
With cores and memory decoupled, I might worry, that we could become
more attractive to VOs to run their highmem jobs - and we starve in the
end there and have cores idleing, that are not accounted (and cause
discussions later on...)
Probably the primary 'issue' is, that AFAIS cores are somewhat the base
currency - in the end the 'relevant' pi charts are just about the
delivered core scaled walltime :-/

Cheers,
  Thomas

On 31/07/2020 20.58, Bockelman, Brian wrote:
> Hi Thomas,
> 
> We do not normalize incoming requirements.
> 
> In your example, I'm not sure if I'm following the benefit.  You are suggesting changing:
> 
> 1 core / 8GB -> 4 core / 8 GB
> 
> Right?  To me, in that case, you now have 3 idle cores inside the job - guaranteed to not be used - rather than 3 idle cores in condor which possibly are not used unless another VO comes in with odd requirements.
> 
> Now, some sites *do* charge for jobs according to both memory and CPU.  So, in your case of 1 core / 2GB being nominal, they would charge the user's fairshare for 4 units if the user submitted a 1 core / 8 GB job.
> 
> Or am I looking at this from the wrong direction?
> 
> Brian
> 
>> On Jul 31, 2020, at 5:02 AM, Thomas Hartmann <thomas.hartmann@xxxxxxx> wrote:
>>
>> Hi all,
>>
>> on your CondorCEs, do you normalize incoming jobs for their core/memory
>> requirements?
>>
>> Thing is, that we normally assume a ratio of ~ 1core/2GB memory.
>> Now let's say a user/VO submits jobs with a skewed ration like
>> 1core/8GB. Which would probably lead to draining for memory and leave a
>> few cores idle.
>> So, I had been thinking, if it might make sense to rescale a job's core
>> or memory requirements in a transform to get the job close to the
>> implicitly assumed core/mem ratio.
>>
>> Does that make sense? ð
>>
>> Cheers,
>>  Thomas
>>
>> _______________________________________________
>> HTCondor-users mailing list
>> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
>> subject: Unsubscribe
>> You can also unsubscribe by visiting
>> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>>
>> The archives can be found at:
>> https://lists.cs.wisc.edu/archive/htcondor-users/
> 
> 
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/
>

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature