Re: [HTCondor-users] job transform: change requirement condition?

[John M Knoeller <johnkn@xxxxxxxxxxx>]

I've been thinking about at this, and I don't think you can do this with a job transform using the job router style syntax 

My reasoning is this:

eval_set_Requirements  will *always* set Requirements to a single literal value, it will not be set it to an expression

set_Requirements can set Requirements to an expression, but that expression would have to use a combination of eval, unparse and regexps.  And the resulting expression would be impossible to analyze and probably break job autoclustering. 

HOWEVER....
If you use the internal transform syntax.  You can use temporary variables and the implicit parsing of expressions
that the internal syntax uses to get the result you want. 

so, something like this might work

JOB_TRANSFORM_NAMES = $(JOB_TRANSFORM_NAMES), FOO 
JOB_TRANSFORM_FOO @=end 
    # only match jobs that require SL6
    REQUIREMENTS regexp("OpSysAndVer\s*=\?*=\s*\"SL6\"", unparse(My.Requirements))

    # unparse the Requirements expression, modify it and save the result as a temporary variable (not saved in the job)
    EVALMACRO temp regexps("OpSysAndVer\s*=\?*=\s*\"SL6\"", unparse(My.Requirements), "OpSysAndVer==\"CentOS7\"")

    # set requirements to the result of expanding the temporary variable and then parsing it as an expression
    SET Requirements = $(temp)
@end

The transform does not begin with [  so  HTCondor will assume it's in internal syntax,
which is like config or submit language and does textual substitution using $() expansion

You can run 

   condor_transform_ads -help rules

for a description of the syntax, and you can test your syntax using condor_transform_ads, putting the transform into a .rules file
and transforming a job read from a file, use -verbose argument to see the transformation. 

-tj

-----Original Message-----
From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Thomas Hartmann
Sent: Thursday, October 11, 2018 9:01 AM
To: htcondor-users@xxxxxxxxxxx
Subject: Re: [HTCondor-users] job transform: change requirement condition?

Hi Thomas F. and Michael,

many thanks for the clues! I think got a bit closer on the thing ;)

Probably I misunderstand regexps() (and struggle with the ClassAd syntax):
does regexps() replaces the whole target string or just the matching
sub-string when it matches the string...?

Thing is, that a transform like [1] matches an input requirement like
  Requirements = ( OpSysAndVer =?= "SL6" ) && ( TARGET.Arch == "X86_64" )
but the resulting requirement is the whole target replaced by the substitute
  Requirements = "OpSysAndVer == CentOS7"
also: it's a string :(

I tried already to explicitly
  eval_set_requirement = eval( regexps(...) )
or to
  set_requirement = eval(regexps(...))
but to no avail.

Additionally, I struggle still with the regexp evaluation in Condor (and
in general) [2]

Maybe somebody as another nudge into the right direction for me? ;)

Cheers and thanks,
  Thomas


[1]
JOB_TRANSFORM_NAMES = $(JOB_TRANSFORM_NAMES) FOOFORM
JOB_TRANSFORM_FOOFORM @=end
[
# first check if we have an SL6 job before proceeding
Requirements = regexp("OpSysAndVer\\s*==\\s*\"SL6\"",
unparse(Requirements));
# and if so, scan for the expression and replace it
eval_set_Requirements = regexps("OpSysAndVer == \"SL6\"",
unparse(Requirements), "OpSysAndVer == CentOS7");
]
@end


[2]
(unfortunately, I am not a regexpert :-[ )

I would like to match '==' as well as '=?=', e.g.,
  > OpSysAndVer == "SL6"
or
  > OpSysAndVer =?= "SL6"

so, I have to match the white spaces around '==' with \s (as far as I
see, backslashes have to be escaped in the syntax)

Next, I have to match none or one occurence of '?'


Sooo, something like '(?)?' (or so decorated with \) should match - but
I have not managed to get something like
  > regexp("OpSysAndVer\\s*=(?)?=\\s*\"SL6\"", unparse(Requirements));
to match both cases '==' and '=?='


Another confusing thing is, that while for the '==' case regexp()
  > regexp("OpSysAndVer\\s*==\\s*\"SL6\"", unparse(Requirements));
seems to match True on 'OpSysAndVer == "SL6"' -- the same expression
with regexps()
  > regexps("OpSysAndVer\\s*==\\s*\"SL6\"", unparse(Requirements),
"OpSysAndVer == CentOS7");
seems not match -- and returns 'error'



On 2018-10-10 18:51, Michael Pelletier wrote:
> Thomas Finnern for the win! Iâve never had occasion to use that function. Page 508 of the manual, in case anyone was wondering.
> 
> Michael V. Pelletier
> Information Technology
> Digital Transformation & Innovation
> Integrated Defense Systems
> Raytheon Company
> 
> From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Finnern, Thomas
> Sent: Wednesday, October 10, 2018 12:44 PM
> To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
> Subject: [External] Re: [HTCondor-users] job transform: change requirement condition?
> 
> Hi Thomas,
> have you tried to replace the BaseRequirements by an unparse(BaseRequirements) within the regexprs ?
> Cheers, Thomas
> 
> Von meinem Mobiltelefon gesendet
> 
> 
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/
>