[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Enforce DiskUsage



yes, stdout and stderr are in the same directory an will be counted as DiskUsage for the job while it is running.

 

-tj

 

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of duduhandelman@xxxxxxxxxxx
Sent: Friday, November 13, 2020 12:12 AM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: Re: [HTCondor-users] Enforce DiskUsage

 

Thank you John.

That's exactly what I need.

 

>Will it include the job output as output?

For example the user requests 1MB of disk. And he wrote nothing to the disk but the std out of his app fill the output log with 2MB of data.

If I remember correctly the output log is in the same execute directory.

And its also depends on the user request when to copy or stream the log.

I think it will solve this case as well.

 

Thank you.

David.

 

 

 

 

 

 

 

Sent from my Mi phone

John M Knoeller <johnkn@xxxxxxxxxxx> wrote on Nov 12, 2020 23:30:<br type="attributi

To put a job on hold when it’s disk usage exceeds its request, add this to the configuration of your execute node

 

 STARTD_JOB_ATTRS =  $(STARTD_JOB_ATTRS) RequestDisk

  DISK_EXCEEDED = (JobUniverse != 13 && DiskUsage =!= UNDEFINED && DiskUsage > RequestDisk)

  use POLICY : WANT_HOLD_IF( DISK_EXCEEDED, 105, disk usage exceeded request_disk )

 

The choice of 105 for the hold subcode is arbitrary here, you can use a different value if you like.

We ignore VM universe (universe 13) because VM disk usage is handled differently than vanilla universe.

 

I don’t understand what you mean by this question

 

> Will it include the job output as well?

 

When a job goes on hold, the output if the job is not returned to the user, is that the question?

 

-tj

 

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of duduhandelman@xxxxxxxxxxx
Sent: Thursday, November 12, 2020 2:25 PM
To: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Subject: [HTCondor-users] Enforce DiskUsage

 

Hi All.

What will be the best way to put job on hold when the DiskUsage >= requested disk?

Will it include the job output as well?

The reason is that one user can fill the disk for all running jobs.

 

Thank you..

David.

 

 

 

Sent from my Mi phone