[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Limiting memory for jobs



> Though I think I would rather the linux kernel limit the amount of
memory that
> can be used by a condor slot, so I would rather pass along that slot
memory
> variable to my user job wrapper and appropriately setup the resource
limit,
> unless there is something really bad with that.

I can't see anything wrong with this. If I wanted this behaviour I'd go
the kernel route as well.

You can tell Condor to put ClassAd information in to the environment of
the job when it is executed. The information can come from the job's
ClassAd or the machine's ClassAd.

For example: I put the slot ID and the full Condor vm+hostname ID into
the environment of all my jobs:

        ABC_REMOTEHOST = vm1@xxxxxxxxxxxxxxxxxxxxxxxxx
        ABC_VMID = 1

I do this by including the following in my submit ticket:

        environment = "ABC_REMOTEHOST=$$(NAME)
ABC_VMID=$$(VIRTUALMACHINEID)"

If you wanted to advertise memory for the slot:

        environment = "SLOT_MEMORY=$$(MEMORY)"

Your wrapper can now use this to call the actual job command with
whatever kernel control you want enforced.

Another option is to fetch the value with condor_status in your wrapper:

        condor_status -direct localhost -f "%d\n" memory

Of course, you'd need to figure out which slot you were running in...

There are probably even more ways than those two mentioned to tackle
this problem. You just need to work out which one works best for your
environment. One requires submit tickets to always have a specific
environment line. The other can be done without submit ticket control,
but requires some additonal logic to work properly in all cases.

- Ian

Confidentiality Notice.
This message may contain information that is confidential or otherwise protected from disclosure. If you are not the intended recipient, you are hereby notified that any use, disclosure, dissemination, distribution,  or copying  of this message, or any attachments, is strictly prohibited.  If you have received this message in error, please advise the sender by reply e-mail, and delete the message and any attachments.  Thank you.