[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Setting requirements based on availability of a package



In the modern versions of HTCondor you can pipe the output of a script which defines key-value pairs to the configuration using "include : script |" (or include command : script"), and add the attribute names in STARTD_ATTRS. Since I started with version 7.8, I still tend to use a OneShot startd cron job instead of an include:

STARTD_CRON_JOBLIST                   = $(STARTD_CRON_JOBLIST) HasTensorFlow
STARTD_CRON_HASTENSORFLOW_MODE           = OneShot
STARTD_CRON_HASTENSORFLOW_RECONFIG_RERUN = True
STARTD_CRON_HASTENSORFLOW_EXECUTABLE     = /usr/bin/python
STARTD_CRON_HASTENSORFLOW_ARGS           = \
" -c 'import pkgutil; print(''HasTensorFlow = True\n--\n'' if pkgutil.find_loader(''tensorflow'') else '''')' "


One advantage of the startd_cron approach is that when certain commands are run by any end-user, the configuration is parsed, which also runs any config-include scripts. The above doesn't do so.

Note the techniques for escaping quotes in the ARGS line - one-liners like this can be vexing to debug if you're not paying close attention. In this example, it leaves the attribute undefined if it's not true.

To view a list of what "has" attributes a machine is advertising, run "condor_status machine -long | grep -I '^has'"

	-Michael Pelletier.


> -----Original Message-----
> From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf
> Of John G Heim
> Sent: Tuesday, August 29, 2017 4:54 PM
> To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
> Subject: [HTCondor-users] Setting requirements based on availability of a
> package
> 
> I have a question from an end-user of our condor cluster. How can you have
> a condor job prefer nodes that have a particular package installed.
>   Something like:
> 
> hasMatlab = $( -e /usr/local/bin/matlab ) requirements = hasMatlab
> --
> --
> John G. Heim; jheim@xxxxxxxxxxxxx; sip://jheim@xxxxxxxxxxxxxxxx
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with
> a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/