Re: [HTCondor-users] htcondor + gpudirect + openmpi

[Michael Pelletier <Michael.V.Pelletier@xxxxxxxxxxxx>]

I've been using the job ad file for non-dynamic values. For dynamic stuff you could use condor_chirp get_job_attr.


     condor_q -jobads $_CONDOR_JOB_AD -autoformat RequestCpus

-Michael Pelletier.


> -----Original Message-----
> From: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] On Behalf
> Of Harald van Pee
> Sent: Monday, September 11, 2017 3:45 PM
> To: htcondor-users@xxxxxxxxxxx
> Subject: Re: [HTCondor-users] htcondor + gpudirect + openmpi
> 
> Hi Jason,
> 
> I think I have done something wrong, or its just working since I have
> installed Mellanox OFED 4.1.
> 
> Up to now I have tested it only with openmpi-2.0.2a1 but at least for this
> version its working if I make a loop over the requested gpus to just get
> more lines in the machine file and for the -n argument I have to multiply
> $_CONDOR_NPROCS with the requested gpus.
> 
> How I get the number of requested gpus in the script?
> At the moment I would parse
> _CONDOR_AssignedGPUs and count them.
> 
> OMP_NUM_THREADS can be used to get the request_cpus value but in general
> this is not the number of mpinodes per node.
> 
> Up to now I just test the cpus but I hope I can start with real gpu jobs
> soon.