Re: [HTCondor-users] condor_userprio

[Alessandra <afortiorama@xxxxxxxxx>]

HI Todd,

The original configuration comes from RAL and it does have group GROUP_PRIO_FACTOR and group GRUOP_QUOTAS_DYNAMIC for each group and subgroup. I started to look into it because the original algorythm mapped VO names to group_names and some of the VO names have dots in them clearly making a mess. So after remapping the VO names to something without dots I have a clearer picture in condor_userprio but still wrong numbers.

All setting GROUP_PRIO_FACTOR_<groupname> does is
set the default priority factor for a new user that submits into that
group for the first time.

Â

I don't understand this, isn't an initial condition for each job and then it gets maybe dumped by excessive usage? or really this number applies only to the very first job? In any case shouldn't the value be correctly reported by the tool as configured and not some number that creeped in at some point? I thought this was the meaning of the columns "priority factor" as configured and "effective priority factor" after calculations get applied.

In any case the point is to configure quotas and priorities for each group. I don't mind if they are hierarchical or not (though ATLAS still asks for it) but there are groups with a small quotas and large priority, also there are groups which occasionally will have their priority bumped up (or reduced) quite a lot for a period and from what you are telling me this is not possible in condor.

I've attached my new configuration files as it might help understanding. In this configuration I removed the Owner from ActtGroup as well because they are pool users anyway and I have no say if user jenny has priority over user tom it also simplifies following things. I left it however in ConcurrencyLimits for now.

thanks

cheers

alessandra

On Mon, 27 Aug 2018 at 12:57, Todd Tannenbaum <tannenba@xxxxxxxxxxx> wrote:

On 8/26/2018 8:40 AM, Alessandra wrote:
> Hi,
>
> i'm trying to adjust the fair shares and priorities in condor. One of
> the things that I don't understand is why the condor_userprio tool
> doesn't report the configured priorities but seems to report random numbers.
>
> for example I have condor_userprio reporting
>
> group_atlas.pilot = 10000
> group_atlas.production = 10000
>
> But the priorities configured are
>
> GROUP_PRIO_FACTOR_group_atlas =Â 10.0
> GROUP_PRIO_FACTOR_group_atlas.pilot =Â 10.0
> GROUP_PRIO_FACTOR_group_atlas.production =Â 10.0
>
> it really should be 10 even if I didn't explicitely declared them
> because the group_atlas is 10. 10k is the default and 1000 I'm not sure
> where it comes from.

Hi Alessandra,

Do the below groups have a group quota? In other words, do you have in
your config a GROUP_NAMES entry and for each group a GROUP_QUOTA_* entry ?

If so, please be aware that these hierarchical groups themselves do not
have a priority factor. The historical fair share scheduling in HTCondor
is only applied to users, not to groups, so all the parameters dealing
with historical fair share scheduling (like real user priority,
effective user priority, and priority factor) have no meaning with
respect to groups. All setting GROUP_PRIO_FACTOR_<groupname> does is
set the default priority factor for a new user that submits into that
group for the first time. I am guessing this is not what you thought it
did.

Instead of historical fair share, the group scheduling is controlled
solely by GROUP_SORT_EXPR, which defaults to "starvation group order."
That is, the group whose current usage is the smallest fraction of its
quota goes first, then the next, and so on.

I realize I did not directly answer you question(s) below, but given the
above information, I am guessing that you really didn't mean to apply a
priority factor to groups in the first place. Perhaps if you told us
your desired scheduling policy someone could better assist...

Hope the above helps,
regards,
Todd

--

Well you'll still need a tray. (Eddie Izzard)

Attachment: 11_fairshares.config
Description: Binary data

Attachment: 00_config_local.config
Description: Binary data