Mailing List Archives

Public Access

 		[Computer Systems Lab]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Missing `x509UserProxyVOName` ClassAd in Condor 10.0.3

Hi Jose,
AFAIS the token issuer ad should be somewhat a good identifier for the VO. I.e., you could maybe write a transform matching & selecting on 
  AuthTokenIssuer
and inject the corresponding VO into x509UserProxyVOName, if it does not exists.
As us, we see a mixed back of GSI and token authz jobs [1], so that one might need to be careful not accidentially miss a case. (being a bit worried about occasional token only jobs and the still large number of pure GSI jobs) 

Cheers,
  Thomas

[1]
> condor_ce_q  -af AuthTokenIssuer  x509UserProxyVOName  | sort | uniq -c
    548 https://atlas-auth.web.cern.ch/ atlas
      4 https://atlas-auth.web.cern.ch/ undefined
    474 https://cms-auth.web.cern.ch/ cms
      3 https://cms-auth.web.cern.ch/ undefined
   4497 undefined belle
      6 undefined desy
     51 undefined ilc
    280 undefined lhcb
    672 undefined ops



On 05/05/2023 10.18, Jose Caballero wrote:

Hi Maarten,

Thanks a lot for the explanation.
Would it be possible to replicate the old functionality with a JOB_TRANSFORM ? 

Cheers,
Jose
El vie, 5 may 2023 a las 9:00, Maarten Litmaath (<Maarten.Litmaath@xxxxxxx <mailto:Maarten.Litmaath@xxxxxxx>>) escribiÃ: 

    Hi JosÃ,
    in 10.x there is no code that looks into the VOMS extensions that an
    X509 proxy may have
    and hence there are no variables defined anymore for the VO and the
    FQANs.

    We will need to decide on sustainable ways for the accounting to
    keep working...


    ------------------------------------------------------------------------
    *From:* HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx
    <mailto:htcondor-users-bounces@xxxxxxxxxxx>> on behalf of Jose
    Caballero <jcaballero.hep@xxxxxxxxx <mailto:jcaballero.hep@xxxxxxxxx>>
    *Sent:* Friday, May 5, 2023 9:33 AM
    *To:* HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx
    <mailto:htcondor-users@xxxxxxxxxxx>>
    *Cc:* condor-users@xxxxxxxxxxx <mailto:condor-users@xxxxxxxxxxx>
    <condor-users@xxxxxxxxxxx <mailto:condor-users@xxxxxxxxxxx>>
    *Subject:* Re: [HTCondor-users] Missing `x509UserProxyVOName`
    ClassAd in Condor 10.0.3
    Hi,

    if I understand correctly what I see, the classAd mentioned by Tom,
    "x509UserProxyVOName", is added to the jobs at the schedd level.
    Picking one random job on a schedd 9.0.5, this is the submit file
    from the CE middleware (ARC) [1] and these are the classad of the
    submitted job [2].
    So clearly the classAds x509* have been added by our local Schedd.

    However, on a schedd 10.0.3, some of those classAds are missing [3].

    I have downloaded the code from GITHUB, and a simple grep gives me
    the same results for the main branch and tag V9_0_5.
    Also, the classAd x509UserProxyVOName is still mentioned in the
    documentation.
    So I am quite lost as well. Why suddenly the jobs submitted from
    schedd 10.x are missing these classads?

    Any comment/question is more than welcome.

    Cheers,
    Jose

    [1]
    [root@arc-ce04
    PG3NDmbdVE3nE6QDjqmt6UqoABFKDmABFKDmhZLKDmABFKDmpuXrkm]# cat
    condorjob.jdl
    # HTCondor job description built by arex
    Executable = condorjob.sh
    Input = /dev/null
    Log =
    /var/spool/arc/grid00/PG3NDmbdVE3nE6QDjqmt6UqoABFKDmABFKDmhZLKDmABFKDmpuXrkm/log
    Output =
    /var/spool/arc/grid00/PG3NDmbdVE3nE6QDjqmt6UqoABFKDmABFKDmhZLKDmABFKDmpuXrkm.comment
    Error =
    /var/spool/arc/grid00/PG3NDmbdVE3nE6QDjqmt6UqoABFKDmABFKDmhZLKDmABFKDmpuXrkm.comment
    +NordugridQueue = "EL7"
    Description = gridjob
    Universe = vanilla
    Notification = Never
    Requirements = (NumJobStarts == 0) && ( (OpSys == "LINUX" &&
    OpSysMajorVer >= 7) )
    Priority = 0
    x509userproxy =
    /var/spool/arc/grid00/PG3NDmbdVE3nE6QDjqmt6UqoABFKDmABFKDmhZLKDmABFKDmpuXrkm/user.proxy
    request_cpus = 1
    request_memory=4000
    +JobMemoryLimit = 4096000
    should_transfer_files = YES
    When_to_transfer_output = ON_EXIT_OR_EVICT
    Transfer_input_files =
    /var/spool/arc/grid00/PG3NDmbdVE3nE6QDjqmt6UqoABFKDmABFKDmhZLKDmABFKDmpuXrkm
    Periodic_remove = (JobStatus == 1 && NumJobStarts > 0) ||
    ((ResidentSetSize isnt undefined ? ResidentSetSize : 0) >
    JobMemoryLimit)
    Queue

    [2]
    [root@arc-ce04
    PG3NDmbdVE3nE6QDjqmt6UqoABFKDmABFKDmhZLKDmABFKDmpuXrkm]# condor_q -l
    2479042 | grep ^x509
    x509userproxy =
    "/var/spool/arc/grid00/PG3NDmbdVE3nE6QDjqmt6UqoABFKDmABFKDmhZLKDmABFKDmpuXrkm/user.proxy"
    x509UserProxyEmail = "lb.pilot@xxxxxxx <mailto:lb.pilot@xxxxxxx>"
    x509UserProxyExpiration = 1683605339
    x509UserProxyFirstFQAN = "/lhcb/Role=pilot/Capability=NULL"
    x509UserProxyFQAN = "/DC=ch/DC=cern/OU=Organic
    Units/OU=Users/CN=lbpilot/CN=693025/CN=Robot: LHCb
    pilot,/lhcb/Role=pilot/Capability=NULL,/lhcb/Role=NULL/Capability=NULL"
    x509userproxysubject = "/DC=ch/DC=cern/OU=Organic
    Units/OU=Users/CN=lbpilot/CN=693025/CN=Robot: LHCb pilot"
    x509UserProxyVOName = "lhcb"

    [3]
    [root@arc-ce-test01 ~]# condor_history -l 605625.0 | grep ^x509
    x509UserProxyEmail = "Andrea.Sciaba@xxxxxxx
    <mailto:Andrea.Sciaba@xxxxxxx>"
    x509UserProxyExpiration = 1682927827
    x509userproxy =
    "/var/spool/arc/grid05/ZPsKDmZFHD3n61QDjqWNiMpoABFKDmABFKDmAaFKDmAEFKDmDzgJen/user.proxy"
    x509userproxysubject = "/DC=ch/DC=cern/OU=Organic
    Units/OU=Users/CN=sciaba/CN=430796/CN=Andrea Sciaba"




    El mar, 2 may 2023 a las 10:33, Thomas Hartmann
    (<thomas.hartmann@xxxxxxx <mailto:thomas.hartmann@xxxxxxx>>) escribiÃ:

        Hi Thomas,

        from Condor 10 on GSI is not supported anymore but only token authz.
        Also IIRC has ATLAS recently switched Harvester submission to
        Condor 10
        as well, so that their jobs do not get submitted anymore with
        X509 ads.

        Probably the only option on the midterm run would be to add
        cases for
        routes, that evaluate the Auth* ads similar as for X509 ads.

        Cheers,
         Â ÂThomas

        On 02/05/2023 10.07, Thomas Birkett - STFC UKRI via
        HTCondor-users wrote:
         > Hi Condor community,
         >
         > I hope you are all keeping well, hopefully a simple fix but Iâve
         > recently upgraded our test Condor pool from 9.0.15 to 10.0.3
        (LTS) and I
         > notice that jobs no longer show the ClassAd
        âx509UserProxyVONameâ. The
         > following x509 classads are present when running a `condor_q
        -l *jobid*`
         >
         > x509UserProxyEmail
         >
         > x509UserProxyExpiration
         >
         > x509userproxy
         >
         > x509userproxysubject
         >
         > however, ` x509UserProxyVOName` is missing.
         >
         > This is a problem for us as a large proportion of our Job
        Transforms use
         > this missing ClassAd `x509UserProxyVOName`. Downgrading to
        Condor
         > 9.0.15, the ClassAd is then applied to new incoming jobs. Any
        help in
         > debugging this issue would be gratefully received.
         >
         > Many thanks,
         >
         > *Thomas Birkett*
         >
         > Senior Systems Administrator
         >
         > Scientific Computing Department
         >
         > Science and Technology Facilities Council (STFC)
         >
         > Rutherford Appleton Laboratory, Chilton, Didcot
         > OX11 0QX
         >
         > signature_609518872
         >
         >
         > _______________________________________________
         > HTCondor-users mailing list
         > To unsubscribe, send a message to
        htcondor-users-request@xxxxxxxxxxx
        <mailto:htcondor-users-request@xxxxxxxxxxx> with a
         > subject: Unsubscribe
         > You can also unsubscribe by visiting
         > https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
        <https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users>
         >
         > The archives can be found at:
         > https://lists.cs.wisc.edu/archive/htcondor-users/
        <https://lists.cs.wisc.edu/archive/htcondor-users/>
        _______________________________________________
        HTCondor-users mailing list
        To unsubscribe, send a message to
        htcondor-users-request@xxxxxxxxxxx
        <mailto:htcondor-users-request@xxxxxxxxxxx> with a
        subject: Unsubscribe
        You can also unsubscribe by visiting
        https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
        <https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users>

        The archives can be found at:
        https://lists.cs.wisc.edu/archive/htcondor-users/
        <https://lists.cs.wisc.edu/archive/htcondor-users/>

    _______________________________________________
    HTCondor-users mailing list
    To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx
    <mailto:htcondor-users-request@xxxxxxxxxxx> with a
    subject: Unsubscribe
    You can also unsubscribe by visiting
    https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
    <https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users>

    The archives can be found at:
    https://lists.cs.wisc.edu/archive/htcondor-users/
    <https://lists.cs.wisc.edu/archive/htcondor-users/>


_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature