Re: [Condor-users] How to 'dynamically' set the VM_MEMORY macro?

[Matthew Farrellee <matt@xxxxxxxxxx>]

Rob wrote:
> Jason Stowe wrote:
>> For more information you can see the manual:
>> http://www.cs.wisc.edu/condor/manual/v7.2/2_5Submitting_Job.html#2421
>> "An example of a job that uses this syntax may be a job that wants to
>> know how much memory it can use. The application cannot detect this
>> itself, as it would potentially use all of the memory on a multi-slot
>> machine. So the job determines the memory per slot, reducing it by 10%
>> to account for miscellaneous overhead, and passes this as a command
>> line argument to the application. In the submit description file will
>> be:
>>
>> arguments=--memory $$([TARGET.Memory * 0.9]) "
>>
>> In your case you'll want to use an expression like:
>> $$([0.6*(Target.Memory - Target.RESERVED_MEMORY)])
> 
> 
> Thank you, but this is not what I want to modify.
> 
> When condor starts its processes (master, startd etc.) on the pool PCs,
> the value for VM_MEMORY has to bet set to a fixed number; this is the
> maximum amount of memory available for a virtual machine.
> 
> (I think the arguments-construct allows me to reduce this amount further
> in the submit file. But that's not what I want.)
> 
> The value of VM_MEMORY defaults to 256 MB, irrespective of the
> amount of total physical memory in the pool PC.
> And this value is set when the master service starts.
> 
> I'd like to set VM_MEMORY to a certain percentage of the total physical
> memory in the pool PC, in much the same way as the VM_MAX_NUMBER
> defaults to $(NUM_CPUS):
> 
> VM_MAX_NUMBER = $(NUM_CPUS)
> VM_MEMORY = $(MEMORY) * 0.9
> 
> The first line is fine, because the number of CPUs is available as the macro
> NUM_CPUS. But the second line fails, because there is apparently no
> macro that tells me the total amount of physical RAM in the pool PC.
> 
> I'd think: if the number of CPUs is available, then why is not the total
> amount of physical RAM???
> 
> Thanks!
> 
> Rob.

Rob,

There are a few things here. First, we could probably make VM_MEMORY
default to a more sane value of the detected amount of memory on the
system. Additionally, the VM_MEMORY macro could (should!?) be changed to
allow it to be an expression. Third, your last thought is actually a
very good one. There are some other cases where having access to
detected resources are beneficial.

Interested/able to lend some effort to these changes?

Best,


matt