Re: [HTCondor-users] how to get current 'fair share' quota/weight of a group?

[Todd Tannenbaum <tannenba@xxxxxxxxxxx>]

On 9/26/2016 10:33 AM, Thomas Hartmann wrote:
> Hi Iain,
>
> thanks for the info.
>
> But while we are running on 8.5.1, I do not find any ads of an
> 'Accounting' type?

Short answer:

You need v8.5.3+ to see the accounting info in the collector, and 
starting with v8.5.4+ the condor_userprio tool queries the collector 
instead of the negotiator.

Longer answer:

We try hard to make the Version History section of the Manual pretty 
comprehensive.  I find it useful to go to Manuals page at
  http://htcondor.org/manual/
then click on "Version History" for the series I care about (v8.5.x, 
v8.4.x, whatever) and see what changed.  Doing this followed by Ctrl-F 
(find in my browser) for the word "accounting", I found

Version 8.5.4

- condor_user_prio now queries the condor_collector for accounting 
information by default, when appropriate. This should be much faster 
than the older way of querying the condor_negotiator. The old path is 
still available by passing the -negotiator option to the tool. (Ticket 
#5508).


Version 8.5.3

- The negotiator now forwards accounting information to the collector, 
where it can be easily queried and monitored. (Ticket #5491).


Clicking on the "Ticket XXX" entries next to items in the Version 
History takes you to the associated development ticket in the HTCondor 
Wiki.  In the ticket you can see details incl discussion/remarks on the 
feature, design documents (for more complex changes), changes made to 
the HTCondor Manual, code changes, etc.

Hope the above helps
Todd



> Skimming through all(?) ads, followingly I do not see neither related
> ads, e.g.,
>   > condor_status -pool condor01 -any -long  | grep Weighted | cut -d "
> " -f 1 | sort | uniq
> WeightedIdleJobs
> WeightedRunningJobs
>   > condor_status -pool condor01 -any -long  | grep Accumulated | cut -d
> " " -f 1 | sort | uniq
>  -
>
> Are we missing maybe something in our setup?
>
> Cheers and thanks,
>   Thomas
>
> On 2016-09-26 16:53, Iain Bradford Steers wrote:
> > Hi Thomas,
> >
> > Which version of condor are you running on your central manager?
> >
> > In the 8.5 series the negotiator now sends it's accounting information to the collector as an ad per item in the tree.
> >
> > Alternatively using the python bindings you can instantiate a Negotiator object and use .getPriorities().
> >
> > The Collector is obviously much faster.
> >
> > Cheers, Iain
> >
> > You can see the list with condor_status  -any -const 'MyType =?= "Accounting"'
> > For example:
> >
> > Parts of the tree also have a couple of others like ConfigQuota set.
> >
> > UnchargedTime = -6
> > LastUsageTime = 1474901308
> > AuthenticatedIdentity = "condor@fsauth"
> > IsAccountingGroup = false
> > PriorityFactor = 100000.0
> > MyType = "Accounting"
> > AccountingGroup = "group_u_X"
> > Name = "group_u_x"
> > WeightedAccumulatedUsage = 128064692224.0
> > LastUpdate = 1474901308
> > WeightedResourcesUsed = 7677.0
> > ResourcesUsed = 7677
> > TargetType = "none"
> > AccumulatedUsage = 127910363136.0
> > LastHeardFrom = 1474901320
> > BeginUsageTime = 1447772083
> > DaemonStartTime = 1473746649
> > UpdateSequenceNumber = 0
> > Priority = 937499392.0
> > WeightedUnchargedTime = -6.0
> > ________________________________________
> > From: HTCondor-users [htcondor-users-bounces@xxxxxxxxxxx] on behalf of Thomas Hartmann [thomas.hartmann@xxxxxxx]
> > Sent: 26 September 2016 16:45
> > To: HTCondor-Users Mail List
> > Subject: [HTCondor-users] how to get current 'fair share' quota/weight of a     group?
> >
> > Hi all,
> >
> > on a user's question [1], I am currently looking for a ClassAd with the
> > current weighting/fair share quota of a given group?
> >
> > I.e.,
> > we have several dynamic quota groups. If I remember correctly, by
> > default the resource usage over the last three days is taken into
> > account for weighting quotas.
> > So, I have been skimming, so far unsuccessfully, through
> >   -state
> >   -collector
> > looking for an ad with a flat value of the current quota.
> > But is there actually such a 'scalar' ad?
> >
> > If not, I assume, that the quota is calculated by the collector each
> > negotiation cycle anew for any queued job with its resource requests
> > (including ConcurrencyLimits, ChildAccountingGroup,...)?
> > And if so, is there an 'easy' algorithm to calculate oneself the
> > collector's hypothetical current quota for a job against everybody else
> > being queued?
> >
> > Cheers and thanks,
> >   Thomas
> >
> >
> > [1]
> > who would like to try to 'safe up' its fair share quota for some more
> > higher priority work load - which is from my point of view really *not*
> > a good approach, i.e., folding some oscillating behaviour into the
> > system,...
> > _______________________________________________
> > HTCondor-users mailing list
> > To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> > subject: Unsubscribe
> > You can also unsubscribe by visiting
> > https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> >
> > The archives can be found at:
> > https://lists.cs.wisc.edu/archive/htcondor-users/
>
>
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/


--
Todd Tannenbaum <tannenba@xxxxxxxxxxx> University of Wisconsin-Madison
Center for High Throughput Computing   Department of Computer Sciences
HTCondor Technical Lead                1210 W. Dayton St. Rm #4257
Phone: (608) 263-7132                  Madison, WI 53706-1685