Re: [HTCondor-users] HTCondor - Slurm integration

[Carl Edquist <edquist@xxxxxxxxxxx>]

Hi Asvija,

Brian asked me to look into this - sorry for the delay getting back to 
you.

The mappings I find based on the condor 8.8.4 version of slurm_submit.sh 
are:

         "BatchProject" ->
         #SBATCH -A $bls_opt_project

         "BatchRuntime" ->
         #SBATCH -t $((bls_opt_runtime / 60))

         "RequestMemory" ->
         #SBATCH --mem=${bls_opt_req_mem}

         "Queue" ->
         #SBATCH -p $bls_opt_queue

         "NodeNumber" ->
         #SBATCH -N $bls_opt_mpinodes

Carl

On Thu, 5 Sep 2019, Asvija B wrote:

> Hi Brian,
>
> Condor version is 8.8.4
>
>
> Thanks and regards,
>
> Asvija
>
> On 9/5/2019 2:33 AM, Brian Lin wrote:
>> Hi Asvija,
>>
>> Unfortunately, there isn't much in terms for documentation but I could
>> give you a mapping if you give me the version of HTCondor you're running.
>>
>> Thanks,
>> Brian
>>
>> On 8/19/19 12:12 AM, Asvija B wrote:
>>> Thanks a lot Brian... I am able to see the +remote_NodeNumber getting
>>> translated properly.
>>>
>>> Can you also please indicate the corresponding directives for other
>>> SLURM related attributes as well (like --nodes, ntasks etc.)
>>>
>>> It would be great if you can point me to some documentation related to
>>> this info..
>>>
>>> Additionally, the slurm_submit.sh file from BLAH's github directory (
>>> https://github.com/prelz/BLAH/blob/master/src/scripts/slurm_submit.sh
>>> ) has additional capabilities of GPU support and MIC support.  Do we
>>> have any documentation which points to the corresponding Condor
>>> directives for these ?
>>>
>>> Thanks again for the information.
>>>
>>> Regards,
>>>
>>> Asvija
>>>
>>>
>>> On 8/16/2019 8:53 PM, Brian Lin wrote:
>>>> Hi Asvjia,
>>>>
>>>> You'll want to specify '+remote_NodeNumber' in your original grid job
>>>> submit file. However, you should note that the Slurm directives we set
>>>> will be changing in future releases of HTCondor 8.9 to the following:
>>>>
>>>> "#SBATCH --nodes=1"
>>>> "#SBATCH --ntasks=1"
>>>> "#SBATCH --cpus-per-task=$bls_opt_mpinodes"
>>>>
>>>> - Brian
>>>>
>>>> On 8/13/19 12:32 AM, Asvija B wrote:
>>>>> Dear Condor users,
>>>>>
>>>>> We are planning to use HT-Condor for submitting jobs to some of our
>>>>> SLURM managed clusters.  As I digged into the documentation, I
>>>>> understood that HT-Condor uses BLAH GAHP for supporting job submission
>>>>> to SLURM.
>>>>>
>>>>> We are interested in submitting MPI jobs to SLURM  through HT-Condor.
>>>>> In this regard, I am unable to look at the configuration parameters in
>>>>> the condor submission script for indicating MPI related information
>>>>> (for eg. number of nodes etc.)
>>>>>
>>>>> I have seen the script file
>>>>> $CONDOR_HOME/libexec/glite/bin/slurm_submit.sh .  It does include
>>>>> statements with   $bls_opt_mpinodes  which translate to "SBATCH -N "
>>>>> directives.   However I am not clear about the equivalent condor
>>>>> directives that will result in the proper SLURM directives. Hence it
>>>>> would be great if any of the SLURM users can comment on this.
>>>>>
>>>>>
>>>>> Thanks and regards,
>>>>>
>>>>> Asvija B
>>>>>
>>>>>
>>>>> 
> ------------------------------------------------------------------------------------------------------------
>>>>>
>>>>>
>>>>> [ C-DAC is on Social-Media too. Kindly follow us at:
>>>>> Facebook: https://www.facebook.com/CDACINDIA & Twitter: @cdacindia ]
>>>>>
>>>>> This e-mail is for the sole use of the intended recipient(s) and may
>>>>> contain confidential and privileged information. If you are not the
>>>>> intended recipient, please contact the sender by reply e-mail and
>>>>> destroy
>>>>> all copies and the original message. Any unauthorized review, use,
>>>>> disclosure, dissemination, forwarding, printing or copying of this
>>>>> email
>>>>> is strictly prohibited and appropriate legal action will be taken.
>>>>> 
> ------------------------------------------------------------------------------------------------------------
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> HTCondor-users mailing list
>>>>> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx
>>>>> with a
>>>>> subject: Unsubscribe
>>>>> You can also unsubscribe by visiting
>>>>> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>>>>>
>>>>> The archives can be found at:
>>>>> https://lists.cs.wisc.edu/archive/htcondor-users/
>>>>
>>> 
> ------------------------------------------------------------------------------------------------------------
>>>
>>> [ C-DAC is on Social-Media too. Kindly follow us at:
>>> Facebook: https://www.facebook.com/CDACINDIA & Twitter: @cdacindia ]
>>>
>>> This e-mail is for the sole use of the intended recipient(s) and may
>>> contain confidential and privileged information. If you are not the
>>> intended recipient, please contact the sender by reply e-mail and destroy
>>> all copies and the original message. Any unauthorized review, use,
>>> disclosure, dissemination, forwarding, printing or copying of this email
>>> is strictly prohibited and appropriate legal action will be taken.
>>> 
> ------------------------------------------------------------------------------------------------------------
>>>
>>>
>>
>>
>
> ------------------------------------------------------------------------------------------------------------
> [ C-DAC is on Social-Media too. Kindly follow us at:
> Facebook: https://www.facebook.com/CDACINDIA & Twitter: @cdacindia ]
>
> This e-mail is for the sole use of the intended recipient(s) and may
> contain confidential and privileged information. If you are not the
> intended recipient, please contact the sender by reply e-mail and destroy
> all copies and the original message. Any unauthorized review, use,
> disclosure, dissemination, forwarding, printing or copying of this email
> is strictly prohibited and appropriate legal action will be taken.
> ------------------------------------------------------------------------------------------------------------
>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/