On 12/26/2013 3:57 PM, Rita wrote:Warning - this is off the top of my head, but I think the following would work or at least point you in the right direction. Also I am assuming you are using HTCondor 8.0 or above.
I have a job with request_memory = 256 Megabytes but if I go over it will
get held according to my SYSTEM_PERIODIC_HOLD policy. I would like to
automatically triple the request_memory and then release the job. Is that
possible to do?
I would expect your condor_config file has entries like the following:
# Tell the schedd to hold jobs that have been restarted more than
# 10 times, or if it uses more memory than it requested
SYSTEM_PERIODIC_HOLD = JobRunCount > 10 || MemoryUsage > RequestMemory
# Set a specific HoldReasonSubCode so if the job uses
# more memory than request, so we can identify this particular
# hold reason in automatic release expressions. With the below
# _expression_, the job's HoldReasonSubCode will be 1 if
# the job is on hold due to memory issues, and 0 otherwise.
SYSTEM_PERIODIC_HOLD_SUBCODE = MemoryUsage > RequestMemory
Then in your job submit file you could try something like:
executable = foo
# HoldReasonSubCode of 26 means SYSTEM_PERIODIC_HOLD became true,
# and HoldReasonSubCode of 1 is configured in condor_config to mean
# job was put on hold due to memory usage.
periodic_release = HoldReasonCode =?= 26 && HoldReasonSubCode =?= 1
# Request 256 MB ram unless MemoryUsage (from a previous run) is
# already defined, in which case triple it for the next try.
request_memory = ifthenelse(isUndefined(MemoryUsage),256,3*MemoryUsage)
Hope the above makes sense - the manual has documentation on all the above knobs and job classad attributes.
Note that if you have control of the configuration of the execute machines, consider having the execute machine itself put the job on hold if memory usage exceeds requested memory. This way the job will be put on hold much sooner after memory usage is exceeded (perhaps even instantly). For some HOWTO recipes, see
HTCondor-users mailing list
To unsubscribe, send a message to firstname.lastname@example.org with a
You can also unsubscribe by visiting
The archives can be found at: