[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[HTCondor-users] Taking action on job breaching scratch space limit



Hello Experts,

I am trying to take action on the job using more scratch space than allocated.ÂÂ

CondorDiskSpace attribute is coming from the dynamic script which is called through condor cron. Script is included before putting this configuration.Â

CondorDiskSpace = $(CondorDiskSpace:1024)
DiskPerCore = Â$(CondorDiskSpace) / $(NUM_CPUS)
chunksdisk = ifThenElse( RequestDisk <= ($(DiskPerCore) * RequestCpus), quantize(RequestCpus, {1}), quantize(RequestDisk, {$(DiskPerCore)}) / $(DiskPerCore))
MODIFY_REQUEST_EXPR_REQUESTDISK = $(chunksdisk) * $(DiskPerCore)

Commented the last two lines in above configuration and added following to keep the test case simple.Â

MODIFY_REQUEST_EXPR_REQUESTDISK = $(DiskPerCore) / 160
ÂSTARTD_JOB_ATTRS = $(STARTD_JOB_ATTRS) TowerTeam DiskUsageÂÂ

Confirmed from the output of condor_who -l that Disk and DiskUsage appears in output. However, still using scratchÂspace more than allocated,Âthe job keeps on running.ÂÂ

DiskUsageBreach = (DiskUsage > Disk)
WANT_HOLD = ($(PREEMPT) || $(DiskUsageBreach))

How can we put the job on hold in this scenario once the job uses more disk space than allocated from scratch directory?Â

Thanks & Regards,
Vikrant Aggarwal