[HTCondor-users] assigned gpus oddity?

[Michael Di Domenico <mdidomenico4@xxxxxxxxx>]

we've switched over from some custom gpu syntax to the new condor
style syntax for gpu's.  i'm having an issue with the AssignedGPUs
classad however.

we're running a mix of 8.2 and 8.4 on linux x86_64 on top of rhel 6

i followed the wiki page below

https://htcondor-wiki.cs.wisc.edu/index.cgi/wiki?p=HowToManageGpus

I can see the GPU's in the machine classads, my issue comes when i
actually run a job

using this sample submit

executable = blah
arguments = $$(AssignedGPUs)
request_gpus=1
request_cpus=1
queue 1

i can see in my environment

CUDA_VISABLE_DEVICES=0
GPU_DEVICE_ORDINAL=0
_CONDOR_AssignedGPUs=CUDA0

but the argument to my program $((AssignedGPUs)) or $1 in my case (its
a shell script) is

CUDA0,CUDA1,CUDA2,CUDA3

Shouldn't the AssignedGPUs argument only be the one i was assigned?

we're setup with fully dynamic slots, meaning a node has a single
dynamic partitionable slot, where 100% of the cpu/memory/gpu are set

did i set something up wrong?