First of all, sorry for my late reply as I got occupied in some family emergencies.
That folder âlibexecâ does not exist.
Everything actually worked when I ran things form remote server. But when i run it from localhost (my own computer) then I see this â../libexecâ missing error.
That is odd. The errors from your previous setup/test attempts ("Invalid qos specificationâ) indicate it was finding the slurm_submit.sh script before. The location of the slurm_*.sh scripts was moved in recent versions, but a consistent installation of Bosco
on the Slurm machine shouldnât be confused about where some of its files are located.
On the Slurm machine, can you check whether $(HOME)/bosco/glite/libexec exists? If is does, what files are under it?
If it doesnât exist, running this command should provide a quick workaround for the error youâre seeing:
ln -s bin $(HOME)/bosco/glite/libexec
can you please tell me why it is looking in the wrong directory for slurm_submit.sh file (as mentioned in my last email) and how can I fix it?
Umet is talking about components in HTCondor-CE, which is similar to Bosco, but is installed by the administrator of the Slurm cluster for use by all remote users. The Job Router is not part of Bosco.
But the suggestions about sacctmgr may be an alternative to manually editing Boscoâs slurm_submit.sh.
Where are we supposed to specify this âJOB_ROUTER_ENTRIESâ parameter?
Also, I donât see a folder âcondorâ under /usr/libexec/ directory. Could you please comment on that?
unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with
can also unsubscribe by visiting
archives can be found at:
The SLURM partition can be specified in the htcondor job routing via the set_default_queue parameter (see below):
GridResource = "batch slurm";
set_default_queue = "grid";
AFAIK there is no way to specify the âqos in the job routing. One way would be to patch the /usr/libexec/condor/glite/bin/slurm_submit.sh blahp script and add it there.
However you can define a DefaultQOS on account/association level using sacctmgr which will be used when you donât define a âqos during submission:
sacctmgr modify account myAccount set DefaultQOS='grid
The above statement only works if the account has access to the QOS. You can set this with: sacctmgr modify account myaccount set QOS='debug,gridâ
You must also make sure that the partition allows the QOS.
This is specified in /etc/slurm/slurm.conf on the controller.
You can check this with: scontrol show part
Check the âAllowQosâ parameter in the output if the QOS is included or the value is set to âALLâ
Vienna BioCenter (GMI, IMP, IMBA)
I am not really sure how to do that. I will explore how to run a job directly to slurm and will get back to you with the output.
Meanwhile, if you know any relevant documentation, kindly share.
BOSCO and HTCondor donât use the -q/--qos command-line option when submitting a job to Slurm. There is a way to have them set the -p/--partition option, but the bosco_cluster tool doesnât use it when testing a new setup.
Can you try submitting a job directly to Slurm on the cluster login node without using the -p or -q options of sbatch?
Thanks for your concerns,
I will look into the qos documentation. Meanwhile, would expect any SLURM expert to chip in.
hopefully someone with slurm experience can jump in here (I better stick to my alternative HTC facts) :D :D :D
As far as I know the partition you use is defined with a list of possible qos (default,debug) for ex. in the submit you can request the qos with the '-q' option.
scontrol show part <partname>
It looks like your job coming from the HTC side of things is requesting a qos that is not defined for the partition you want to use.
Unfortunately that is even thinner ice for me than usualy on this list :(
Also I did not fully get the concept of slurm at the time and started using condor instead ;)
But you seem to be on the right track, and should check the documentation about qos and how to define them for partiions and users alike:
Building 02b, Room 009
Now I ran it on the server (where bosco node is hosted) and it shows me the following output
User Def Acct Admin Cluster Account Partition Share MaxJobs MaxNodes MaxCPUs MaxSubmit MaxWall MaxCPUMins QOS Def QOS
---------- ---------- --------- ---------- ---------- ---------- --------- ------- -------- -------- --------- ----------- ----------- -------------------- ---------
hbaig pi-mendes None xanadu pi-mendes 1 general,himem,speci+
Could you please tell me if it tells something meaningful which needs to be fixed.
acctmgr show user <user> withassoc