Re: [HTCondor-users] persistency of generic ClassAds

[Thomas Hartmann <thomas.hartmann@xxxxxxx>]

Hi Todd,

many thanks for the links! With it, I managed to get my ClassAds injected :)

One minor correction needed compared to the wiki [1] was to add the job
to the STARTD_CRON_JOBLIST but not the STARTD_CRON_NAME.
afais STARTD_CRON_NAME had been removed >7.7 [2]

From the docs I assume, that no crons are possible for masters but only
for startds and schedds?
I was thinking about cases, where one may wants to add some generic
monitoring information for both, plain workers and plain submitters, and
thus trying to go via the master? But I guess in the end it is not much
of a hassle just to reuse the same config for a startd and a schedd setup.

Cheers and thanks,
  Thomas

[1]
https://htcondor-wiki.cs.wisc.edu/index.cgi/wiki?p=HowToInsertClassAdIntoMachineAds

[2]
http://research.cs.wisc.edu/htcondor/manual/v8.5/3_3Configuration.html#param:SubsysExprs


On 2016-08-02 19:02, Todd Tannenbaum wrote:
> Hi Thomas,
> 
> Off the top of my head (i.e. I didn't test it), I think you want to put
> the "- update:true" line at the end of your script, so that it is the
> last line going to stdout instead of the first line.
> 
> Apologies that the documentation on all of the STARTD_CRON stuff is
> weak; if fact, we have a ticket to improve this soon at
>   https://htcondor-wiki.cs.wisc.edu/index.cgi/tktview?tn=5403
> 
> In the meantime, you may find the last remark at the end of the
> following ticket helpful as it has some examples:
>   https://htcondor-wiki.cs.wisc.edu/index.cgi/tktview?tn=4398
> Also see some examples in the following HOWTO :
> 
> https://htcondor-wiki.cs.wisc.edu/index.cgi/wiki?p=HowToInsertClassAdIntoMachineAds
> 
> 
> Hope this helps,
> best regards,
> Todd
> 
> 
> On 8/2/2016 10:29 AM, Thomas Hartmann wrote:
>> Hi Brian et al.,
>>
>> I am still struggling to get my own cron/ClassAd extension up and
>> running.
>>
>> Following section 4.4.3, I am trying with a dummy script [1] to push
>> some ClassAds to the startd's Ads [2].
>> I would have expected the script to be run each minute and updating the
>> collector (with update:true set). However, the ClassAds keep their
>> default values set in the configuration, when querying the values from .
>>
>> What am I missing here, that the cron job's ClassAd values are not
>> propagated?
>>
>> Cheers and thanks,
>>    Thomas
>>
>> [1] (chmod a+x)
>>> /etc/condor/monitoring/test
>> #!/bin/sh
>>
>> echo -e  "- update:true
>> STARTD_CRON_TEST_KILL = FALSE
>> STARTD_CRON_TEST_ARGS = Something Else
>> STARTD_CRON_TEST_SLOTS = -1
>> STARTD_CRON_TEST_JOB_LOAD = 0.345
>> STARTD_CRON_TEST_SQUID = tentacle
>> STARTD_CRON_TEST_SQUIRREL = tail"
>>
>> [2]
>>> /etc/condor/config.d/99ClassAdHookstest.conf
>> STARTD_CRON_JOBLIST = $(STARTD_CRON_JOBLIST) test
>> STARTD_CRON_TEST_MODE = Periodic
>> STARTD_CRON_TEST_RECONFIG_RERUN = True
>> STARTD_CRON_TEST_PREFIX = test_
>> STARTD_CRON_TEST_EXECUTABLE = /etc/condor/monitoring/test
>> SCHEDD_CRON_TEST_PERIOD = 1m
>> STARTD_CRON_TEST_KILL = Maybe
>> STARTD_CRON_TEST_ARGS = abc 123 5556
>> STARTD_CRON_TEST_SLOTS = 3
>> STARTD_CRON_TEST_JOB_LOAD = 0.76
>> STARTD_CRON_TEST_SQUID = purple
>>
>> On 2016-07-27 20:46, Brian Bockelman wrote:
>>> Hi Thomas,
>>>
>>> Is this related to your previous question?  In that case, I might
>>> have sent you a bit in the wrong direction.
>>>
>>> The easiest way to incorporate this sort of information is to get it
>>> the collector via *startd ad* or *master ad* (as you explore below,
>>> it’s kind of a pain to put it into a completely separate ad).
>>>
>>> Putting extra information into an existing ad is far easier and
>>> (fairly) well documented:
>>>
>>> http://research.cs.wisc.edu/htcondor/manual/v8.5/3_3Configuration.html#param:SubsysExprs
>>>
>>>
>>> The STARTD_EXPRS or MASTER_EXPRS mechanism is going to be good for
>>> fairly static information (think: kernel version).
>>>
>>> For more dynamic information, you’ll want to use the STARTD_CRON, as
>>> documented here:
>>>
>>> http://research.cs.wisc.edu/htcondor/manual/v8.5/4_4Hooks.html#SECTION00543000000000000000
>>>
>>>
>>> At our site, we use this to report the health of CVMFS into the
>>> startd classad.  That allows us to have the START expression turn to
>>> UNDEFINED if CVMFS breaks.
>>>
>>> Brian
>>>
>>>> On Jul 27, 2016, at 5:32 AM, Thomas Hartmann
>>>> <thomas.hartmann@xxxxxxx> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> I some questions about their persistency
>>>>
>>>> - afais ClassAd key:value pairs cannot be updated individually but the
>>>> whole set has to be rewritten, or?
>>>>    I.e., after I set some generic ClassAds [1,2], I updated(?) the
>>>> generic set setting only one key-value-pair and followingly all
>>>> previously set but not mentioned again ClassAds became undefined [3,4]
>>>>
>>>> - how persistent are generic ClassAds? Will they survive a reboot of
>>>> the Collector, i.e., are they written to disk somewhere?
>>>>
>>>> - can I start with default values set by a node?
>>>>    I.e., Since I found no matching hits in a quick google search, I
>>>> already tried (without success), if one can set ClassAds similar to
>>>> adding daemon attributes [5]?
>>>>    - would that make actually sense? Or would any manually set ClassAd
>>>> be overwritten again the next time a node's master speaks with the
>>>> collector?
>>>>
>>>> Cheers and thanks,
>>>>   Thomas
>>>>
>>>> [1]
>>>>> test_batch0930.ad
>>>> MyType = "Generic"
>>>> TestKernel = "2.3.5"
>>>> TestDoStuff = True
>>>> TestStatusThing = "bar"
>>>> Name = "batch0930.desy.de"
>>>> Machine = "batch0930.desy.de"
>>>>
>>>> [2]
>>>>> condor_status -generic -constraint 'regexp(".*batch0930.*", Name)'
>>>>> -long
>>>> LastHeardFrom = 1469614306
>>>> AuthenticatedIdentity = "unauthenticated@unmapped"
>>>> MyAddress = "<131.169.56.33:0>"
>>>> UpdatesHistory = "00000000000000000000000000000000"
>>>> UpdatesLost = 0
>>>> UpdatesSequenced = 0
>>>> TestStatusThing = "bar"
>>>> Machine = "batch0930.desy.de"
>>>> TestDoStuff = true
>>>> TestKernel = "2.3.5"
>>>> UpdatesTotal = 6
>>>> Name = "batch0930.desy.de"
>>>> MyType = "Generic"
>>>>
>>>> [3]
>>>>> test_batch0930_2.ad
>>>> MyType = "Generic"
>>>> TestStatusThing = "ALERT"
>>>> Name = "batch0930.desy.de"
>>>> Machine = "batch0930.desy.de"
>>>>
>>>>
>>>> [4]
>>>>> condor_status -generic -constraint 'regexp(".*batch0930.*\.desy\.de",
>>>> Name)' -af name TestStatusThing TestDoStuff TestKernel
>>>> batch0930.desy.de ALERT undefined undefined
>>>>
>>>>
>>>> [5] # (GENERIC_ATTRS certainly wrong...)
>>>>> /etc/condor/config.d/99test.conf
>>>> GENERIC_TestKernel = False
>>>> GENERIC_TestDoStuff = False
>>>> GENERIC_TestStatusThing = False
>>>> TestKernel = False
>>>> TestDoStuff = False
>>>> TestStatusThing = False
>>>> GENERIC_ATTRS = TestKernel, TestDoStuff, TestStatusThing,
>>>> $(GENERIC_ATTRS)
>>>> GENERIC.SETTABLE_ATTRS_ADMINISTRATOR = TestKernel, TestDoStuff,
>>>> TestStatusThing, $(GENERIC.SETTABLE_ATTRS_ADMINISTRATOR)
>>>>
>>>> _______________________________________________
>>>> HTCondor-users mailing list
>>>> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx
>>>> with a
>>>> subject: Unsubscribe
>>>> You can also unsubscribe by visiting
>>>> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>>>>
>>>> The archives can be found at:
>>>> https://lists.cs.wisc.edu/archive/htcondor-users/
>>>
>>>
>>> _______________________________________________
>>> HTCondor-users mailing list
>>> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx
>>> with a
>>> subject: Unsubscribe
>>> You can also unsubscribe by visiting
>>> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>>>
>>> The archives can be found at:
>>> https://lists.cs.wisc.edu/archive/htcondor-users/
>>>
>>
>>
>>
>> _______________________________________________
>> HTCondor-users mailing list
>> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx
>> with a
>> subject: Unsubscribe
>> You can also unsubscribe by visiting
>> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>>
>> The archives can be found at:
>> https://lists.cs.wisc.edu/archive/htcondor-users/
>>
> 
>

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature