[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [HTCondor-users] "include command : " in configuration files
- Date: Tue, 7 Nov 2017 17:33:00 +0000
- From: John M Knoeller <johnkn@xxxxxxxxxxx>
- Subject: Re: [HTCondor-users] "include command : " in configuration files
> If I go to a worker node (startd), as admin, and call "condor_config_val -startd -dump", I immediately see the new value created from an included script
The startd will not load a new config unless something is *else* is forcing the startd to reconfig. condor_config_val will not force a reconfig, and "condor_config_val -startd -dump" will cause condor_config_val print values from the startd's (possibly stale) config.
HOWEVER! each time you run a tool, like condor_q, condor_submit, or condor_config_val. the tool will parse the config and run any scripts as part of the tool initialization. (this is needed so that the tools can find the collector, etc).
So a config script WILL run as part of "condor_config_val -startd -dump", but the output of condor_config_val will be the values it got from the startd, not the values it got by parsing the config itself.
From: Oliver Freyermuth [mailto:freyermuth@xxxxxxxxxxxxxxxxxx]
Sent: Monday, November 6, 2017 6:57 PM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>; John M Knoeller <johnkn@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] "include command : " in configuration files
Am 06.11.2017 um 23:58 schrieb John M Knoeller:
> The problem here is that you should not let just anyone make the cached config files.
> You should instead make sure that ONLY a HTCondor daemon creates that cache file, all other config readers should just use the cache file.
> The reason for this your config files provide a way to get HTCondor daemons (probably running as root!) to run arbitrary code. This means
> that if any of your config files (including the cache files) is writable by non-root then you have created a root exploit.
Agreed. That's why I would prefer if the cache file was generated by HTCondor and not by my script itself,
which is why I suggested the "maxage" setting.
> You should ONLY use cached config files with the following pattern
> if $(IsMaster)
> include ifexist command into $(LOCAL_DIR)/stuff.config : perl $(LOCAL_DIR)/stuff.pl
> include ifexist : $(LOCAL_DIR)/stuff.config
> You could use $(IsStartd) or $(CondorIsAdmin) instead. These all guarantee that only a daemon running as root
> (or Admin on Windows) will create the cache file.
That's a nice best-practice, thanks!
> Of course, once you do this it becomes clear why having a timeout on the cache file doesn't do much good. The master
> doesn't re-parse the config unless you reconfig or restart it - so a timeout isn't going to do much good unless you have
> a cron job reconfig-ing the master regularly - and if you do you might as well have the cron job delete the stuff.config file
> just before it reconfigs the master.
That would be the way I would have gone, and would have assumed things to work from the docs - but then, apparently, the answer to my original question is wrong.
My original question was, "which daemon" "how often" re-evaluates the configuration.
If I go to a worker node (startd), as admin, and call "condor_config_val -startd -dump", I immediately see the new value created from an included script
(if no cache file is used).
How does this work? Why does this cause the startd to re-parse the configuration?
Surely there's no daemon being restarted when I type "condor_config_val -startd -dump" - or does this actually restart the startd?
Cheers and many thanks,
> -----Original Message-----
> From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf Of Oliver Freyermuth
> Sent: Friday, November 3, 2017 2:15 PM
> To: htcondor-users <htcondor-users@xxxxxxxxxxx>
> Subject: Re: [HTCondor-users] "include command : " in configuration files
> Dear Michael,
> many thanks for your reply!
> Am 03.11.2017 um 20:05 schrieb Michael Pelletier:
>> It does indeed run every time the config is read, including when condor_submit or condor_config_val are run by any user.
>> This caused some issues with condor_gpu_discovery in an earlier version which emitted a CUDA error which tripped up the configuration. I don't recall my support ticket number on that, but it should show up in the recent release notes.
>> The workaround I did is to set up any config-included script to cache the persistent results and display the cached value if it's fresh enough.
> This is a very good idea to allow to define the caching period ourselves!
> I'll go with a similar solution, then, once the script gets more complicated (as of now, it just does a "stat()" and a "cat" of a file).
> Since it appears this is not only affecting me, but also others,
> it might be a good idea though (for the long run) to have something like (with maxage 1200 specifying a 20 minute caching):
> include ifexist command into $(LOCAL_DIR)/stuff.config maxage 1200 : perl $(LOCAL_DIR)/stuff.pl
> in HTCondor (which will do just the same we are right now doing with external workarounds).
> Since HTCondor has to check for the existence of the cache-file in any case, this should be very straightforward to implement in the code.
> Is this mailing list also the suitable place to collect such ideas / feature requests?
> Cheers and many thanks,
>> -Michael Pelletier.
>>> -----Original Message-----
>>> From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf
>>> Of Oliver Freyermuth
>>> Sent: Friday, November 03, 2017 1:05 PM
>>> To: htcondor-users <htcondor-users@xxxxxxxxxxx>
>>> Subject: [External] [HTCondor-users] "include command : " in configuration
>>> Dear HTCondor experts,
>>> I am now using a statement like:
>>> include command : /some/shell/script
>>> (i.e. without cache) in a configuration file to set a variable which is
>>> later used to define SINGULARITY_IMAGE_EXPR, i.e. the images used for
>>> We want these to be updated according to an external file stored on CVMFS,
>>> which is edited on a different host, so the "polling" is actually
>>> I am wondering, though, how often "/some/shell/script" is actually called.
>>> The documentation only states:
>>> "If the into option is not used, the command line will be executed every
>>> time the configuration file is referenced. "
>>> From this, I would have guessed this happens as often as a daemon is
>>> restarted (in this case, the condor_startd).
>>> However, it seems that this happens more often - does it happen each time
>>> *any* configuration parameter is queried?
>>> Is there a regular, minimal refresh interval?
>>> Many thanks and all the best,
>> HTCondor-users mailing list
>> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
>> subject: Unsubscribe
>> You can also unsubscribe by visiting
>> The archives can be found at:
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> The archives can be found at: