[HTCondor-users] APPEND_REQUIREMENTS vs JOB_TRANSFORM

[frederic.boismenu@xxxxxxxxxxx]

â
Hello HTCondor Users,

We are investigating different ways to "inject" a simple requirement clause into every users' job, something like "My.WorkerAttribute =?= Target.WorkerAttribute".

Our first attempt was to settle on APPEND_REQUIREMENTS which is attractive because it seems to be a less CPU intensive approach for our submit boxes.

It all worked well, until we tried to submit jobs directly as classad with Python binding. We are building the job definition as a classad.ClassAd(), 
which is then htcondor.Schedd().submit() 'ed.

To our surprise the APPEND_REQUIREMENTS does not seem to be honored in this case. But if we submit a job defined as htcondor.Submit() it is honored.

According to our tests, job transforms always works whether submitting from a classad or from a Submit object.

Even more surprisingly, while testing different configurations we noticed the APPEND_REQUIREMENTS behavior was retained by our client Python REPL even after restarting condor daemons,
which makes me suspect APPEND_REQUIREMENTS is actually implemented on client/library side at the job description parsing time, and that condor_schedd daemon is not even aware of it?

Could you please shade some light on the job submission actually works?  when and where macros are evaluated (such as $(Cluster)) ?
What's the relationship between the high level description level submit file format and actual classad sent to the schedd ? It seems there's a lot of magic actually happening in the client actually?

A diagram like the following which describing step 2. in details would be awesome :) 
http://research.cs.wisc.edu/htcondor/CondorWeek2009/condor_presentations/admin_tutorial/â


Many thanks in advance,
Fred