Re: [HTCondor-users] HTCondor - Slurm integration

Mailing List Archives Public Access	UW Madison Computer Sciences Department Computer Systems Lab

Brian--can you give us an idea of when the changes in SLURM parameters are likely to hit the 8.9 stream? I have been putting off testing the initial +remote_ structure and if it is going to change again I will just wait until the new changes hit.

Also--are there any plans to add the capacity to put an arbitrary line into the slurm submit script?

For instance--these are all SLURM parameters that we have to add manually on the remote BLAHP installation in our slurm_local_submit_options.sh at NERSC and change with a text editor every time they change, which is not infrequently.

If we had a utility to just put an arbitrary line or lines into the slurm submit on the

other end that would be very helpful, particularly since NERSC keeps changing what

some of these parameters are called, and adding extra ones.

echo "#SBATCH --account=m2612"

#echo "#SBATCH --reservation=xrootd_debug"

echo "#SBATCH -N 1"

echo "#SBATCH -q regular"

echo "#SBATCH -C knl,cache,quad"

echo "#SBATCH --image=cmssw/cms:rhel7"

echo "#SBATCH -L cscratch1,cvmfs"

echo "#SBATCH --module=cvmfs"

echo "#SBATCH --volume=\"/global/cscratch1/sd/uscms/node_cache:/tmp:perNodeCache

=size=680G\""

echo "#SBATCH -t 48:00:00"

Steve Timm

Hi Asvjia,

You'll want to specify '+remote_NodeNumber' in your original grid job
submit file. However, you should note that the Slurm directives we set
will be changing in future releases of HTCondor 8.9 to the following:

"#SBATCH --nodes=1"
"#SBATCH --ntasks=1"
"#SBATCH --cpus-per-task=$bls_opt_mpinodes"

- Brian

On 8/13/19 12:32 AM, Asvija B wrote:
> Dear Condor users,
>
> We are planning to use HT-Condor for submitting jobs to some of our
> SLURM managed clusters. As I digged into the documentation, I
> understood that HT-Condor uses BLAH GAHP for supporting job submission
> to SLURM.
>
> We are interested in submitting MPI jobs to SLURM through HT-Condor.
> In this regard, I am unable to look at the configuration parameters in
> the condor submission script for indicating MPI related information
> (for eg. number of nodes etc.)
>
> I have seen the script file
> $CONDOR_HOME/libexec/glite/bin/slurm_submit.sh . It does include
> statements with $bls_opt_mpinodes which translate to "SBATCH -N "
> directives. However I am not clear about the equivalent condor
> directives that will result in the proper SLURM directives. Hence it
> would be great if any of the SLURM users can comment on this.
>
>
> Thanks and regards,
>
> Asvija B
>
>
> ------------------------------------------------------------------------------------------------------------
>
> [ C-DAC is on Social-Media too. Kindly follow us at:
> Facebook: https://urldefense.proofpoint.com/v2/url?u=https-3A__www.facebook.com_CDACINDIA&d=DwIGaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=10BCTK25QMgkMYibLRbpYg&m=q2gG0V4FhTqYAG6riOlFKJpm7dwloFmuPpbuTCogyp4&s=WnaUhNJZ_vgT_m08fHdM3cXF1hn_iMdFPGDV8JDpQk4&e= & Twitter: @cdacindia ]
>
> This e-mail is for the sole use of the intended recipient(s) and may
> contain confidential and privileged information. If you are not the
> intended recipient, please contact the sender by reply e-mail and destroy
> all copies and the original message. Any unauthorized review, use,
> disclosure, dissemination, forwarding, printing or copying of this email
> is strictly prohibited and appropriate legal action will be taken.
> ------------------------------------------------------------------------------------------------------------
>
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx
> with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.cs.wisc.edu_mailman_listinfo_htcondor-2Dusers&d=DwIGaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=10BCTK25QMgkMYibLRbpYg&m=q2gG0V4FhTqYAG6riOlFKJpm7dwloFmuPpbuTCogyp4&s=raFMuym3y0ubG-Xe1_AOB2phvdN_jgZ5qzb-mcjOJqg&e=
>
> The archives can be found at:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.cs.wisc.edu_archive_htcondor-2Dusers_&d=DwIGaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=10BCTK25QMgkMYibLRbpYg&m=q2gG0V4FhTqYAG6riOlFKJpm7dwloFmuPpbuTCogyp4&s=1r1k2ZHQ3jvA5ARPReqZlWz5j_KAqpFMFavZr13THw0&e=

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.cs.wisc.edu_mailman_listinfo_htcondor-2Dusers&d=DwIGaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=10BCTK25QMgkMYibLRbpYg&m=q2gG0V4FhTqYAG6riOlFKJpm7dwloFmuPpbuTCogyp4&s=raFMuym3y0ubG-Xe1_AOB2phvdN_jgZ5qzb-mcjOJqg&e=

The archives can be found at:
https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.cs.wisc.edu_archive_htcondor-2Dusers_&d=DwIGaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=10BCTK25QMgkMYibLRbpYg&m=q2gG0V4FhTqYAG6riOlFKJpm7dwloFmuPpbuTCogyp4&s=1r1k2ZHQ3jvA5ARPReqZlWz5j_KAqpFMFavZr13THw0&e=

Mailing List Archives

Public Access

Re: [HTCondor-users] HTCondor - Slurm integration