[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] chaining CondorCE routes?



Hi Thomas,

Unfortunately, routes are exclusive and terminal so your job will only apply the rules in ROUTE_TEST_1 unless you have an earlier route that matches in JOB_ROUTE_NAMES (did you mean JOB_ROUTER_ROUTE_NAMES?). This is easier to do in the new job router transform syntax (we're aiming to provide this in 5.1.0): there will be pre-route transforms, post-route transforms, and route transforms each with their own REQUIREMENTS _expression_ to determine whether or not a transform is applied to a given job. So in this system, you could do something like [1] to achieve what you want.

In the meantime, you can either duplicate the rules across routes or use the 'set_' or 'eval_set_' directives to add ClassAd expressions to JOB_ROUTER_DEFAULTS that are conditional upon route 1 + route 2 requirements (https://htcondor-ce.readthedocs.io/en/latest/batch-system-integration/#setting-attributes-for-all-routes).

- Brian

[1]

JOB_TRANSFORM_ROUTE_TEST_1 @=end
  REQUIREMENTS True
  SET GOTROUTE1 True
@end

JOB_TRANSFORM_ROUTE_TEST_2 @=end
  REQUIREMENTS True
  SET GOTROUTE2 True
@end

JOB_ROUTER_PRE_ROUTE_TRANSFORMS = $(JOB_ROUTER_PRE_ROUTE_TRANSFORMS) ROUTE_TEST_1 ROUTE_TEST_2

JOB_ROUTER_ROUTE_Local_Condor @=jrt
  TargetUniverse = 5
@jrt

JOB_ROUTER_ROUTE_NAMES = Local_Condor

On 3/25/21 5:59 AM, Thomas Hartmann wrote:
Hi all,

is it actually possible to chain routes and apply multiple to a matching job?

I have been trying something like [1], where I define multiple routes and which should apply to all jobs (~`Requirements = True`)  -  with the intention to modularize rules a bit. But I have not managed to convince the CE to apply all matching(?) routes to a job.
Thing is, that I have also tried to also order the routes, but it seems, that while all the routes are added to the routing table [3] only the first one gets applied [2].

Versions in use are CondorCE 4.4.1 with Condor 8.9.11 [4]

Cheers and thanks for ideas,
  Thomas

[1]
> cat 99_10_routedebug.conf

ROUTETEST1 @=end
[
  TargetUniverse = 5;
  name = "ROUTE_TEST_1";
  Requirements = True;
  set_GOTROUTE1=True
]
@end

ROUTETEST2 @=end
[
  TargetUniverse = 5;
  name = "ROUTE_TEST_2";
  Requirements = True;
  set_GOTROUTE2=True
]
@end

JOB_ROUTER_ENTRIES = $(JOB_ROUTER_ENTRIES) $(ROUTETEST1) $(ROUTETEST2)
JOB_ROUTE_NAMES = $(JOB_ROUTE_NAMES) $(ROUTETEST1) $(ROUTETEST2)


[2]
03/25/21 11:48:08 JobRouter: Checking for candidate jobs. routing table is:
Route Name               Source      Submitted/Max        Idle/Max Throttle Recent: Started Succeeded Failed
ROUTE_TEST_1             entries:1         2/  10000       2/   2000 none               1         0      0
ROUTE_TEST_2             entries:2         0/  10000       0/   2000 none               0         0      0
Local_Condor             entries:0         2/  10000       2/   2000 none               0         0      0


[3]
root@grid-htcondorce-dev: [/etc/condor-ce/config.d] condor_ce_config_val JOB_ROUTE_NAMES JOB_ROUTER_ENTRIES
 [
TargetUniverse = 5;
name = "ROUTE_TEST_1";
Requirements = True;
set_GOTROUTE1=True
] [
TargetUniverse = 5;
name = "ROUTE_TEST_2";
Requirements = True;
set_GOTROUTE2=True
]


[
TargetUniverse = 5;
name = "Local_Condor";
] [
TargetUniverse = 5;
name = "ROUTE_TEST_1";
Requirements = True;
set_GOTROUTE1=True
] [
TargetUniverse = 5;
name = "ROUTE_TEST_2";
Requirements = True;
set_GOTROUTE2=True
]



[4]
condor-8.9.11-1.el7.x86_64
condor-boinc-7.16.11-1.el7.x86_64
condor-classads-8.9.11-1.el7.x86_64
condor-externals-8.9.11-1.el7.x86_64
condor-procd-8.9.11-1.el7.x86_64
htcondor-ce-4.4.1-3.el7.noarch
htcondor-ce-apel-4.4.1-3.el7.noarch
htcondor-ce-bdii-4.4.1-3.el7.noarch
htcondor-ce-client-4.4.1-3.el7.noarch
htcondor-ce-condor-4.4.1-3.el7.noarch
htcondor-ce-view-4.4.1-3.el7.noarch
python2-condor-8.9.11-1.el7.x86_64
python3-condor-8.9.11-1.el7.x86_64


_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/