[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] help/warn user if specified value not string wrapped in double quotes



Thanks for letting us know !!!!


-- 
Christoph Beyer
DESY Hamburg
IT-Department

Notkestr. 85
Building 02b, Room 009
22607 Hamburg

phone:+49-(0)40-8998-2317
mail: christoph.beyer@xxxxxxx


----- UrsprÃngliche Mail -----
Von: Rundall, Jacob D <rundall@xxxxxxxxxxxx>
An: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Gesendet: Sat, 25 Jan 2020 01:43:29 +0100 (CET)
Betreff: Re: [HTCondor-users] help/warn user if specified value not string wrapped in double quotes

unparse(Nodeset) did the trick. I was still left with the possibility that the user may specify a hostname as the value for Nodeset, and that they may do w/o double quotes, e.g.:
$ condor_submit <submit file> -append '+Nodeset=prj-cluster-sub01'

And in that case, unparsed(Nodeset) ends up being "prj - cluster - sub01".

I tried unsuccessfully to get use regex functions (esp. replaceall, regexps) to strip spaces but couldn't make that work. But I browsed through functions and discovered that the following both adds double quotes and then strips spaces:
join(split(unparse(Nodeset)))

So I now have two Job Transforms that do what I want: set the default Nodeset to "NORMAL", if undefined; and wrap in "" (& remove spaces)
JOB_TRANSFORM_NAMES = $(JOB_TRANSFORM_NAMES) DEFNODESET WRAPNODESET
JOB_TRANSFORM_DEFNODESET @=ENDDNS
[
   REQUIREMENTS = ((unparse(Nodeset)) == "") && isUndefined(Nodeset);
   set_Nodeset = "NORMAL";
]
@ENDDNS
JOB_TRANSFORM_WRAPNODESET @=ENDWNS
[
   REQUIREMENTS = ((unparse(Nodeset)) != "") && isUndefined(Nodeset);
   eval_set_Nodeset = join(split(unparse(Nodeset)));
]
@ENDWNS

Thanks much!

Jake
ïOn 1/23/20, 4:30 PM, "HTCondor-users on behalf of Michael Pelletier via HTCondor-users" <htcondor-users-bounces@xxxxxxxxxxx on behalf of htcondor-users@xxxxxxxxxxx> wrote:

    The issue is that while condor_submit is reasonably clever about discerning the data type of various values, because it understands what attribute each key intends to define within the job ClassAd and the target type of that ClassAd, when you use "+" or "MY." prefixes, you are directly specifying the ClassAd attribute, so condor_submit can't sort out the type for you.
    
    For example:
    
    -----
    executable = /bin/sleep
    arguments = 30
    
    +NodeSet=DEBUG
    MY.NodeSetTwo = Normal
    -----
    
    When you do this, you're saying that the "NodeSet" job attribute is equal to the "DEBUG" job attribute (case-insensitive but case-preserving), and that NodeSetTwo is equal to the "Normal" job attribute, which is because they're not "string" types in the ClassAd language due to the lack of double quotes around them, and there's nothing condor_submit can do about it.
    
    [pelletm@engcloud-cm1 ~]$ condor_submit subsleep
    Submitting job(s).
    1 job(s) submitted to cluster 32074.
    [pelletm@engcloud-cm1 ~]$ condor_q -long 32074 | grep Node
    NodeSet = DEBUG
    NodeSetTwo = Normal
    [pelletm@engcloud-cm1 ~]$
    
    This may have the appearance of having worked, but if you "-autoformat" them, the parser will look up the "debug" and "normal" attributes respectively, and they'll both come back undefined as you've seen.
    
    Your case looks like the "unparse(NodeSet)" function call would be useful - if the return from that is an empty string AND NodeSet is undefined, then you can set it to "normal," but if unparse() returns a nonempty string even though NodeSet is undefined, then you'd just set the NodeSet attribute to the string which unparse returned.
    
    
    Michael V. Pelletier
    Information Technology
    Digital Transformation & Innovation
    Integrated Defense Systems
    Raytheon Company
    
    From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Rundall, Jacob D
    Sent: Thursday, January 23, 2020 4:24 PM
    To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
    Subject: [External] [HTCondor-users] help/warn user if specified value not string wrapped in double quotes
    
    We have a custom Nodeset attribute weâd like to be set for each job. If the user does not specify Nodeset weâd like it to be set to "NORMAL". If the user does specify a Nodeset we'd like it to be set as they specified.
    
    A Job Transform like this comes close to doing what we want:
    JOB_TRANSFORM_NAMES = $(JOB_TRANSFORM_NAMES) DEFNODESET
    JOB_TRANSFORM_DEFNODESET @=ENDDNS
       REQUIREMENTS NODESET IS UNDEFINED
       SET NODESET "NORMAL"
    @ENDDNS
    
    In particular, if the user does not specify anything, then Nodeset is set to "NORMAL". And if they specify a string in "", it is preserved, e.g.,
    $ condor_submit <submit file> -append '+Nodeset="DEBUG"'
    This results in Nodeset = "DEBUG" (the job transform is skipped; again, this is desired).
    
    However, if the user attempts to specify Nodeset but passes text w/o using double quotes, it seems to be treated as Undefined and is set to "NORMAL". E.g.,
    $ condor_submit <submit file> -append '+Nodeset=DEBUG'
    Results in Nodeset = "NORMAL" (the job transform takes effect; this is NOT desired).
    
    In this case, we either want to make HTCondor wrap the non-string in "", e.g., DEBUG becomes "DEBUG", or otherwise we probably want to skip the Job Transform and use a Submit Requirement to catch the non-string and tell the user to try again with "".
    
    Note that we have the additional complication that a Nodeset may be a hostname w/ hyphens, e.g., prj-cluster-sub01. I realize that those hyphens are vastly different if interpreted as minus or unary negation as opposed to the hyphen character in a string.
    
    Is there a way to achieve what we're after: Warn/help the user out if they forget to wrap the Nodeset value in "" ?
    
    _______________________________________________
    HTCondor-users mailing list
    To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
    subject: Unsubscribe
    You can also unsubscribe by visiting
    https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
    
    The archives can be found at:
    https://lists.cs.wisc.edu/archive/htcondor-users/


_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/