Re: [HTCondor-users] job_transform and preserving requirements

[Jason Patton <jpatton@xxxxxxxxxxx>]

Hi Stefano,

Job transforms are probably what you want for this. There is an updated syntax that you can use to define each transform that has the added benefit of being able to more easily access already-defined variables (e.g. "$(My.Requirements)"). Following one of your examples, this looks like:

JOB_TRANSFORM_Stor_ams @=end
   Requirements = AcctGroup =?= "ams"
   set Requirements (TARGET.GPFS_AMS =!= False) && $(My.Requirements)
@end

(No more [] blocks and semi-colons needed!)

Jason

On Tue, Jul 7, 2020 at 1:24 PM Stefano Dal Pra <stefano.dalpra@xxxxxxxxxxxx> wrote:

Hello,
htcondor 8.8.9 speaking.

We have several local user groups, each one using a specific filesystem
which is expected to be mounted and available on the execute node.
Example:
-  jobs submitted by users in the "ams" unix group would work with
/storage/gpfs_ams ;
-  jobs submitted by users in the "borexino", "pamela", "rdfa",...,
groups would work with /storage/gpfs_data
-  jobs submitted by users in the  ..., ..., ..., groups would work with
/storage/gpfs_<xyz>

I want to add requirements to these jobs submitted to one schedd, so
that they require a machine where the needed

filesystem is available, which is advertised by these custom classads:
GPFS_AMS  --> true / false / undefined
GPFS_DATA --> true / false / undefined
...
GPFS_<xyz> --> true / false / undefined

My idea would be to use JOB_TRANSFORM somewhat like this:

#this should set AcctGroup
use FEATURE:AssignAccountingGroup($(T1_SHARED_SCRIPT_DIR)/Hgroups.txt)

JOB_TRANSFORM_NAMES = $(JOB_TRANSFORM_NAMES) Stor_ams Stor_data
Stor_name1 Stor_name2
JOB_TRANSFORM_Stor_ams = [ Requirements = AcctGroup =?= "ams";\
set_requirements = (TARGET.GPFS_AMS =!= False) ]
JOB_TRANSFORM_Stor_data = [ Requirements =
StringListMember(AcctGroup,"borexino:pamela:rdfa:<SNIP>",":");\
set_requirements = (TARGET.GPFS_DATA =!= False)]

JOB_TRANSFORM_name1 = [ ....]
JOB_TRANSFORM_name2 = [ ....]

with the problem however that doing so i would override the original
requirements specified by the users.
Is there a way to preserve them?

I see that there is the knob: APPEND_REQUIREMENTS
and i probably could set that using a chain of nested IfThenElse( ...,
..., ...). However i would prefere JOB_TRANSFORM because it seems more
versatile, as i could later add
more constraints if needed (i.e. a runtime limit or memory limit per
group, or so)

Thanks for any suggestion

Stefano
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/