[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Accounting Groups



The accounting_group keyword in the submit file is used to set the AccountingGroup attribute in the job classad.

But it does *not* just get copied into AccountingGroup.  The AccountingGroup job attribute from several  pieces of information including the current username, the value of accounting_group  and the value of the  accounting_group_user keyword.

 

Note that as a general rule submit keywords will have an underscore in the name, job attributes will not, so

accouting_group is a submit keyword,  while AccoutingGroup is a job classad attribute.   Only condor_submit understands

submit keywords,  all of the rest of HTCondor only understands classad attributes.

 

The AccountingGroup attribute in the job (when it exists) is used by the condor_schedd and condor_negotiator as the name

that is charged for useage, and may have a quota.    When a job does not have an AccountingGroup

attribute, then the name used for accounting and quotas is derived from the Owner attribute of the job.

 

condor_submit will set AcctGroup to the literal value of the accounting_group keyword from the submit file.  This is for

your reference, it has no effect on quotas or accounting. 

 

condor_submit will set AcctGroupUser to the literal value of the accouting_group_user keyword from the submit file.  This is

for your reference, it also has no effect on quotas for accounting.

 

-tj

 

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of rmorgan466@xxxxxxxxx
Sent: Saturday, May 2, 2020 6:03 PM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] Accounting Groups

 

Just checked these settings.

 

I have both of them as false.

GROUP_ACCEPT_SURPLUS = false

GROUP_AUTOREGROUP = false

 

When I submit the job I see 

condor_q -l 

AcctGroup = "group_userA"

AcctGroupUser = "userA" 

 

Since this doesnt work I tried it without all these settings.

 

submit file should have

accounting_group=group_a

 

so, what is the difference betwee AcctGroup/AcctGroupUser AND accounting_group? 

 

 

 

On Sat, May 2, 2020 at 4:29 PM Michael Pelletier via HTCondor-users <htcondor-users@xxxxxxxxxxx> wrote:

This may have something to do with your settings of the GROUP_ACCEPT_SURPLUS and GROUP_AUTOREGROUP configuration parameters.

If GAS is true, then a group will be allowed to use more than its quota if there's resources available.

If GA is true, then a submitter from a given group will be reconsidered as a no-group submitter making the jobs eligible for matching even if the group has reached its quota.


Michael V Pelletier
Principal Engineer

Raytheon Technologies
Information Technology
Digital Transormation & Innovation


From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of rmorgan466@xxxxxxxxx
Sent: Saturday, May 2, 2020 1:29 PM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: [External] Re: [HTCondor-users] Accounting Groups

tj (and everyone else):
thankyou. Thats better.

I did something like this now
GROUP_NAMES = group_userA
GROUP_QUOTA_dynamic_group_userA = .5

But when I submit jobs all slots are taken up. I though only 50% of the slots should be occupied. 



On Mon, Apr 27, 2020 at 10:43 AM John M Knoeller <mailto:johnkn@xxxxxxxxxxx> wrote:
So I see a small typo here that is going to prevent the map file from working correctly.
 
The first argument to usermap needs to be the mapfile name,  you are using "AssignGroup",
but the mapfile name is Groups.
 
Also the attribute that you want to set is AccountingGroup, not AcctGroup.  Setting AcctGroupUser
and AcctGroup is optional â it can help you to see what the transform is doing, but these are attributes
that none of the Htcondor daemons care about.  AccountingGroup is the only one that really matters.
 
So your config might look like this.
 
SCHEDD_CLASSAD_USER_MAP_NAMES = $(SCHEDD_CLASSAD_USER_MAP_NAMES) Groups 
CLASSAD_USER_MAPFILE_Groups = /path/to/mapfile 
 
JOB_TRANSFORM_NAMES = AssignGroup 
JOB_TRANSFORM_AssignGroup @=end 
[
copy_Owner="AcctGroupUser";
copy_AccountingGroup="RequestedAcctGroup";
eval_set_AccountingGroup=usermap("Groups",Owner,AccountingGroup);

@end
 
Finally, I see some people already addressed this, but since AcctGroupUser is being used in the transform as a job attribute.
Using it *also* as a group name prefix on the CM is going to cause some confusion.   It might be better to use a different name there, the convention is to just use âgroupâ like this
 
GROUP_NAMES = group_userA
GROUP_QUOTA_group_userA = 100
 
 
Hope this helps.
-tj
 
 
From: HTCondor-users <mailto:htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of mailto:rmorgan466@xxxxxxxxx
Sent: Thursday, April 23, 2020 4:35 PM
To: HTCondor-Users Mail List <mailto:htcondor-users@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] Accounting Groups
 
I was able to get further.  Thanks to this, https://urldefense.proofpoint.com/v2/url?u=https-3A__indico.cern.ch_event_817927_contributions_3570542_attachments_1913668_3162964_ScheddTransforms.pdf&d=DwMFaQ&c=ilBQI1lupc9Y65XwNblLtw&r=EuOhLkDgcSYmjXWaaDqfrVrUkEt6FN-uNf0mksEUAGA&m=giQQH0NulZ2Ww8m6kfdrXkaALJpgvwjdgxsxeX-Cyrc&s=aR41KAAeyB9ddbzXqlABcMNNiIAnFahPmTjW9I5O6iY&e=
 
SCHEDD_CLASSAD_USER_MAP_NAMES = $(SCHEDD_CLASSAD_USER_MAP_NAMES) Groups 
CLASSAD_USER_MAPFILE_Groups = /path/to/mapfile 
 
JOB_TRANSFORM_NAMES = AssignGroup 
JOB_TRANSFORM_AssignGroup @=end 
[ copy_Owner="AcctGroupUser"; copy_AcctGroup="RequestedAcctGroup"; eval_set_AcctGroup=usermap("AssignGroup",AcctGroupUser,AcctGroup); ] 
@end
 
On my CM, I now have
GROUP_NAMES = AcctGroupUser
GROUP_QUOTA_AcctGroupUser_userA = 100
 
That should set userA quota to 100.
 
is this correct? Also, what is the purpose of the mapfile? I know it must exist but not sure why we need it. 
 
On Thu, Apr 23, 2020 at 12:40 PM Rita <mailto:rmorgan466@xxxxxxxxx> wrote:
I set the following below
 
Restarted condor. Submitted a job. And now, I see in my condor_userprio my username and resource used. Is this a good way to validate the change? Now that user accounting groups are squared away, How can I enforce the quota?
 
JOB_TRANSFORM_NAMES = AssignGroup
 
JOB_TRANSFORM_AssignGroup = [ eval_set_AccountingGroup = userMap("Groups",Owner,AccountingGroup); ]
 
 
 
 
 
On Thu, Apr 23, 2020 at 12:19 PM Gregory Thain <mailto:gthain@xxxxxxxxxxx> wrote:

On 4/23/20 10:46 AM, mailto:rmorgan466@xxxxxxxxx wrote:
> Hi.
>
> I was wondering if there were any advancements on how to manage
> account groups. Each user will have an accounting group in my setup. I
> want to explicitly set the number of jobs a user can run. I don't want
> a user to use another accounting group. If a user runs without an
> accounting group, their jobs shouldn't run. Is this possible?


With schedd transforms, the schedd can now automatically assign an
accounting group to a user based on a mapping file the admin sets up. 
Some details are given here, though we hope to improve the documentation:

https://urldefense.proofpoint.com/v2/url?u=https-3A__htcondor.readthedocs.io_en_latest_admin-2Dmanual_policy-2Dconfiguration.html-3Fhighlight-3Dusermap-23job-2Dtransforms&d=DwMFaQ&c=ilBQI1lupc9Y65XwNblLtw&r=EuOhLkDgcSYmjXWaaDqfrVrUkEt6FN-uNf0mksEUAGA&m=giQQH0NulZ2Ww8m6kfdrXkaALJpgvwjdgxsxeX-Cyrc&s=jqyVkNNynj6xRFK6WVaQkeY-5T8ekZ30l3VAm4K2-DY&e=


-greg


_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to mailto:htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.cs.wisc.edu_mailman_listinfo_htcondor-2Dusers&d=DwMFaQ&c=ilBQI1lupc9Y65XwNblLtw&r=EuOhLkDgcSYmjXWaaDqfrVrUkEt6FN-uNf0mksEUAGA&m=giQQH0NulZ2Ww8m6kfdrXkaALJpgvwjdgxsxeX-Cyrc&s=RiXvwvEPbjOvrf-9M9LfQWAiHX-mN4duoEctjS1cJGc&e=

The archives can be found at:
https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.cs.wisc.edu_archive_htcondor-2Dusers_&d=DwMFaQ&c=ilBQI1lupc9Y65XwNblLtw&r=EuOhLkDgcSYmjXWaaDqfrVrUkEt6FN-uNf0mksEUAGA&m=giQQH0NulZ2Ww8m6kfdrXkaALJpgvwjdgxsxeX-Cyrc&s=GIMEBCOfXoJ_OGKr49vk2aTz9TxfA5t_PJ2wmLT0WMc&e=


 
--
--- Get your facts first, then you can distort them as you please.--


 
--
--- Get your facts first, then you can distort them as you please.--
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to mailto:htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.cs.wisc.edu_mailman_listinfo_htcondor-2Dusers&d=DwMFaQ&c=ilBQI1lupc9Y65XwNblLtw&r=EuOhLkDgcSYmjXWaaDqfrVrUkEt6FN-uNf0mksEUAGA&m=giQQH0NulZ2Ww8m6kfdrXkaALJpgvwjdgxsxeX-Cyrc&s=RiXvwvEPbjOvrf-9M9LfQWAiHX-mN4duoEctjS1cJGc&e=

The archives can be found at:
https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.cs.wisc.edu_archive_htcondor-2Dusers_&d=DwMFaQ&c=ilBQI1lupc9Y65XwNblLtw&r=EuOhLkDgcSYmjXWaaDqfrVrUkEt6FN-uNf0mksEUAGA&m=giQQH0NulZ2Ww8m6kfdrXkaALJpgvwjdgxsxeX-Cyrc&s=GIMEBCOfXoJ_OGKr49vk2aTz9TxfA5t_PJ2wmLT0WMc&e=



--
--- Get your facts first, then you can distort them as you please.--

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/


 

--

--- Get your facts first, then you can distort them as you please.--