Re: [HTCondor-users] Programmatic access to match analysis?

[Jose Caballero <jcaballero.hep@xxxxxxxxx>]

Hi

this thread catch my attention.
Question for the experts: is not possible to get the classads
advertised by all the startd's to the collector using the python
bindings?
I see in the docs a line like this

         startd_ads = coll.query(htcondor.AdTypes.Startd)

Would that be the entire list of machine classads?
If yes, then you can do matchmaking for a job ad against that list,
without submitting, right?

Cheers,
Jose


El miÃ., 30 ene. 2019 a las 15:49, Ian McEwen (<mian@xxxxxxxxxxx>) escribiÃ:
>
> This is a decent start! We don't have C++ expertise in-house, so ideally
> this would be accessible some other fairly sane way -- python bindings
> or similar -- rather than making any attempt at parsing condor_q output
> or reimplementing the analysis code ourselves in one language or
> another.
>
> Similarly to what Michael said, it'd also be nice if there's a way to
> query the negotiator for concurrency limits, rather than having
> out-of-band knowledge of what the limits are set to and extracting it
> from the running jobs. But in both of these cases I do at least realize
> the officially-blessed way of doing this would be to use condor_q
> interactively and our system's way of doing things is out of the norm.
>
> Thanks for the response and the pointer to the condor_q code for this!
>
> On Tue, Jan 29, 2019 at 05:43:45PM +0000, John M Knoeller wrote:
> > If you are writing C++ code,  the better-analyze code is in HTCondor source at src\condor_q.V6\jobmatch.cpp. look at the doJobRunAnalysis function.
> >
> > Were you looking for programmatic access in a different language?
> >
> > By the way this code does matchmaking analysis only.  There is currently no code in HTCondor outside of the Negotiator itself that checks concurrency limits -  only the Negotiator knows what limits have been handed out already.
> >
> > -tj
> >
> > -----Original Message-----
> > From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Ian McEwen
> > Sent: Monday, January 28, 2019 6:34 PM
> > To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
> > Subject: [HTCondor-users] Programmatic access to match analysis?
> >
> > Hi;
> >
> > This isn't something I'm expecting to have time to do soon, so I'm just
> > gathering information for now, but I'm wondering if there's an
> > appropriate way to get programmatic access to what `condor_q -analyze`
> > and `-better-analyze` output, especially for jobs that have not yet been
> > submitted.
> >
> > That is, given a job ad (constructed however is appropriate), is there a
> > good way to test, non-interactively and without submitting the ad,
> > whether there is a slot that meets the requirements (or, perhaps, if
> > there could be one with partitionable slot defragmentation?), whether
> > concurrency limits are met, etc.?
> >
> > As many of you probably know, we run a system where the direct
> > interaction with HTCondor (including building job ads and submitting
> > them) is handled by us, so users are using our web interface without
> > needing to know HTCondor itself, but lately we've been adding more
> > features that allow users to select their requirements more
> > specifically, which may increase the possibility that jobs will be
> > unable to match (or wait a long time to do so). I'd like to build
> > something that can give users an idea of whether their job is likely to
> > run soon, and why or why not.
> >
> > Thanks in advance for any help!
> > _______________________________________________
> > HTCondor-users mailing list
> > To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> > subject: Unsubscribe
> > You can also unsubscribe by visiting
> > https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> >
> > The archives can be found at:
> > https://lists.cs.wisc.edu/archive/htcondor-users/
> >
> > _______________________________________________
> > HTCondor-users mailing list
> > To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> > subject: Unsubscribe
> > You can also unsubscribe by visiting
> > https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> >
> > The archives can be found at:
> > https://lists.cs.wisc.edu/archive/htcondor-users/
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/