[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] JOB TRANSFORM Questions



Actually, now that I look at this more closely,  the other way to fix this is to use RequstedAcctGroup
as the 3rd argument to the usermap function, and to duplicate the usermap function as the second argument
in your strcat.  that would get rid of the ordering dependency. 

JOB_TRANSFORM_AssignGroup @=end
[
copy_Owner="TestAcctGroupUser";
copy_TestAcctGroup="RequestedAcctGroup";
set_TestPartition = "group_LOCAL";
eval_set_TestAcctGroup = usermap("Groups",TestAcctGroupUser, RequestedAcctGroup); 
eval_set_TestAccountingGroup = strcat(TestPartition, ".", usermap("Groups",TestAcctGroupUser, RequestedAcctGroup), ".", TestAcctGroupUser);
]
@end

copy operations happen before set operations which happen before eval_set operations. 

And since, you can use $() substitutions in your config files, you can do this make things a bit more readable.

TEST_USERMAP_EXPR = usermap("Groups",TestAcctGroupUser, RequestedAcctGroup)

JOB_TRANSFORM_AssignGroup @=end
[
copy_Owner="TestAcctGroupUser";
copy_TestAcctGroup="RequestedAcctGroup";
set_TestPartition = "group_LOCAL";
eval_set_TestAcctGroup = $(TEST_USERMAP_EXPR); 
eval_set_TestAccountingGroup = strcat(TestPartition, ".", $(TEST_USERMAP_EXPR), ".", TestAcctGroupUser);
]
@end

-tj

-----Original Message-----
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of John M Knoeller
Sent: Wednesday, October 16, 2019 3:12 PM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] JOB TRANSFORM Questions

There isn't any need to use evalset for TestAcctGroup.  If you set it as an expression, it will still get set before
the evalset of TestAccountingGroup, and will thus be defined. 

That being said,  you may not want to shove an expression into the job.  In that case you need to control the order of
the evalset statements.  You can control the order of evaluation by using the native transform syntax directly. 

Have a look at the Schedd log on startup, and you will see that you transform is being converted
into native syntax.    That syntax is evaluated top down, unlike the job-router style syntax that you
are currently using. 

If your transform does not start with [  then it will be parsed as the native syntax. So your config could look like this

JOB_TRANSFORM_AssignGroup @=end
        NAME AssignGroup
        COPY Owner TestAcctGroupUser
        COPY TestAcctGroup RequestedAcctGroup
        SET TestPartition "group_LOCAL"
        EVALSET TestAcctGroup usermap("Groups",TestAcctGroupUser,TestAcctGroup)
        EVALSET TestAccountingGroup strcat(TestPartition, ".", TestAcctGroup, ".",  TestAcctGroupUser)
@end

The native syntax is described in a bit more detail by the condor_transform_ads tool.  run

condor_transform_ads -help rules

to see the rules. 

-tj

-----Original Message-----
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Chris Brew - UKRI STFC
Sent: Wednesday, October 16, 2019 8:47 AM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: [HTCondor-users] JOB TRANSFORM Questions

Hi,

Iâm trying to our horrible nested IfThenElse statements that set up our hierarchical accounting groups with a nice clean JOB_TRANSFORM.

Naively I started with:

 JOB_TRANSFORM_AssignGroup @=end
[
copy_Owner="TestAcctGroupUser";
copy_TestAcctGroup="RequestedAcctGroup";
set_TestPartition = "group_LOCAL";
eval_set_TestAcctGroup = usermap("Groups",TestAcctGroupUser,TestAcctGroup); 
eval_set_TestAccountingGroup = strcat(TestPartition, ".", TestAcctGroup, ".", TestAcctGroupUser);
]
@end

but TestAccountingGroup always got set to âundefinedâ because TestAcctGroup was âundefinedâ when it was evaluated.

Iâve âsolvedâ it by setting TestAccountingGroup in a second transform, but is there a cleaner way of controlling which order the eval_sets get evaluated?

That works to set TestAccountingGroup to the default for that user but if I try to select a non-default group I still get the default:

The config is:

SCHEDD_CLASSAD_USER_MAP_NAMES = $(SCHEDD_CLASSAD_USER_MAP_NAMES) Groups
CLASSAD_USER_MAPFILE_Groups = /etc/condor/maps/local_user_map
# Assign groups automatically
JOB_TRANSFORM_NAMES = $(JOB_TRANSFORM_NAMES) AssignGroup AccountingGroup
JOB_TRANSFORM_AssignGroup @=end
[
copy_Owner="TestAcctGroupUser";
copy_TestAcctGroup="RequestedAcctGroup";
set_TestPartition = "group_LOCAL";
eval_set_TestAcctGroup = usermap("Groups",TestAcctGroupUser,TestAcctGroup);
]
@end

JOB_TRANSFORM_AccountingGroup @=end
[
eval_set_TestAccountingGroup = strcat(TestPartition, ".", TestAcctGroup, ".", TestAcctGroupUser);
]
@end

The map file has:

$ grep brew /etc/condor/maps/local_user_map
 *  brew CMS,ATLAS

The Job definition has:

$ grep TestAcctGroup testjob
+TestAcctGroup         = "ATLAS"

But the eventual job classad has:

RequestedAcctGroup = "ATLAS"
TestAccountingGroup = "group_LOCAL.CMS.brew"
TestAcctGroup = "CMS"
TestAcctGroupUser = "brew"
TestPartition = "group_LOCAL"

Can anyone see what Iâve done wrong?

Many Thanks,
Chris.

--
Dr Chris Brew
Scientific Computing Manager
Particle Physics Department
UKRI - STFC - Rutherford Appleton Laboratory
Harwell Oxford,
Didcot
OX11 0QX
+44 1235 446326 
 


_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/