Re: [HTCondor-users] Dynamically determine the amount of extensible machine resource

[Todd Tannenbaum <tannenba@xxxxxxxxxxx>]

On 5/17/2013 10:23 AM, Zhe Zhang wrote:
> Hi,
>
> I know that HTCondor support extensible machine resource. Links at
> http://spinningmatt.wordpress.com/2012/11/19/extensible-machine-resources/
> explains this.
>
> For example, if you want to advertise GPU as a resource, you can add the
> following in your config files:
>
> MACHINE_RESOURCE_NAMES = GPU
> MACHINE_RESOURCE_GPU = 2
>
> SLOT_TYPE_1 = cpus=100%,auto
> SLOT_TYPE_1_PARTITIONABLE = TRUE
> NUM_SLOTS_TYPE_1 = 1
>
> Now I have some resource, e.g. the bandwidth, which I also want to
> advertise as a machine resource in this way. However, instead of hard
> coding each machine with a specific value for bandwidth, I want it to be
> generalized as for different machine, I could dynamically determine the
> bandwidth resource (I have a piece of code does that) and put it in the
> config file. Is there a way I can do that?

Hi Zhe! Yes, there is a way to do that (a few ways actually).  One way 
off the top of my head would be to use MACHINE_RESOURCE_INVENTORY_<xxx> 
knob.  This specifies a script to run at startup of the startd - the 
stdout of this script should spit out an attribute "Dectected<xxx>=y" 
for the extensible resource plus any other optional characteristics of 
the resource to advertise in the machine classad.

So instead of doing:

  # declare a local resource quantity directly in config
  MACHINE_RESOURCE_railgun = 8

You could do something like this:

  # specify a script to run that returns local resource inventory with
  # other optional attributes to be advertised in slot ads
  MACHINE_RESOURCE_INVENTORY_railgun = /bin/railgun_inventory.sh 8 0.95c

where railgun_inventory.sh look like this:

  #!/bin/sh
  echo "DetectedRailgun = $1"
  echo -e "RailgunMuzzleVelocity = \"$2\""

P.s. If you do not specify "MACHINE_RESOURCE_NAMES", the startd should 
be able to figure it out automatically by searching for all the 
MACHINE_RESOURCE_* and MACHINE_RESOURCE_INVENTORY_* knobs...

Hope the above helps,
regards,
Todd