[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[HTCondor-users] job_transform and preserving requirements



Hello,
htcondor 8.8.9 speaking.

We have several local user groups, each one using a specific filesystem which is expected to be mounted and available on the execute node.
Example:
-  jobs submitted by users in the "ams" unix group would work with /storage/gpfs_ams ; -  jobs submitted by users in the "borexino", "pamela", "rdfa",..., groups would work with /storage/gpfs_data -  jobs submitted by users in the  ..., ..., ..., groups would work with /storage/gpfs_<xyz>

I want to add requirements to these jobs submitted to one schedd, so that they require a machine where the needed

filesystem is available, which is advertised by these custom classads:
GPFS_AMS  --> true / false / undefined
GPFS_DATA --> true / false / undefined
...
GPFS_<xyz> --> true / false / undefined

My idea would be to use JOB_TRANSFORM somewhat like this:

#this should set AcctGroup
use FEATURE:AssignAccountingGroup($(T1_SHARED_SCRIPT_DIR)/Hgroups.txt)

JOB_TRANSFORM_NAMES = $(JOB_TRANSFORM_NAMES) Stor_ams Stor_data Stor_name1 Stor_name2
JOB_TRANSFORM_Stor_ams = [ Requirements = AcctGroup =?= "ams";\
set_requirements = (TARGET.GPFS_AMS =!= False) ]
JOB_TRANSFORM_Stor_data = [ Requirements = StringListMember(AcctGroup,"borexino:pamela:rdfa:<SNIP>",":");\
set_requirements = (TARGET.GPFS_DATA =!= False)]

JOB_TRANSFORM_name1 = [ ....]
JOB_TRANSFORM_name2 = [ ....]

with the problem however that doing so i would override the original requirements specified by the users.
Is there a way to preserve them?

I see that there is the knob: APPEND_REQUIREMENTS
and i probably could set that using a chain of nested IfThenElse( ..., ..., ...). However i would prefere JOB_TRANSFORM because it seems more versatile, as i could later add more constraints if needed (i.e. a runtime limit or memory limit per group, or so)

Thanks for any suggestion

Stefano