[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] gLite/GAHP vs BOSCO



The easiest way to get a working BLAH with HTCondor is to download a release from here: https://research.cs.wisc.edu/htcondor/downloads/
We have both native packages and a binary tarball for most popular linux distros. If those arenât suitable, and you need to build from source, then use the configure_uw script in the root of the source tree to configure cmake. That will build a copy of BLAH along with Condor.

For a remote cluster without admin rights, BOSCO would be the best option if you have a small number of users submitting to the remote cluster (and they all have accounts there). Thereâs a one-time setup step for each user and they will need to specify which jobs are intended for the remote cluster.
With multiple users, GlideIn may be well-suited if you can have a shared account on the remote cluster. All of the GlideIns would run as a single user in the cluster, but could accept jobs from any of several users in the Condor pool. You would want to get permission from the clusterâs admin first.
With a small number of amenable users with accounts on the remote cluster, you could instead submit GlideIns under each their accounts. The GlideIns would be configured to only run jobs from that user.

 - Jaime

On Aug 8, 2019, at 12:42 PM, Koschmieder, Lukas Michael <Lukas.Koschmieder@xxxxxxxxxxxxxx> wrote:

Hi Jaime,
 
thank you very much for this overview. I have a few follow-up questions though.
 
1.       I am actually in the position where I could configure a single machine to be a submit point for both PBS and HTCondor. If I understand correct, all I would have to do in this case is setting up BLAH? Iâve just compiled HTCondor but it appears that my build is missing the BLAH interface. Do I have to compile BLAH separately and then rebuild HTCondor against it?
2.       In the future, I would also like to include a remote cluster. But other than now, I will not have administrative rights there. It appears to me that in this case, GlideIn would be the only option. And Iâm wondering if GlideIn also makes sense in a multi-user setup? Does it handle user mapping?
 
Thanks again and best regards,
Lukas
 
Von: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> Im Auftrag von Jaime Frey
Gesendet: Thursday, August 8, 2019 6:21 PM
An: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>
Betreff: Re: [HTCondor-users] gLite/GAHP vs BOSCO
 
On Aug 7, 2019, at 11:31 AM, Koschmieder, Lukas Michael <Lukas.Koschmieder@xxxxxxxxxxxxxx> wrote:
 
I would like add a PBS cluster to my Condor pool. There are at least two solutions mentioned in the documentation - gLite/GAHP and BOSCO. After quick review, it seems to me that gLite is now obsolete.
 
1.       Would you agree?
2.       Is there any cases in which you would recommend using gLite over BOSCO?
3.       Where would I find the gLite code to be placed in GLITE_LOCATION?
 
gLite/GAHP and BOSCO are two different ways of using the same underlying piece of software, called either âbatch_gahpâ or âblahpâ (depending on where youâre installing HTCondor from). Itâs still maintained (though no longer as part of gLite) and is included in the HTCondor release. Itâs one of several ways to incorporate a PBS cluster into your HTCondor environment.
 
If you want user jobs in HTCondor to be directly submitted into PBS, that usually means that users need to indicate in their submit files which jobs are intended to run in PBS (instead of on HTCondor execute nodes). If you can configure a single machine to be a submit point for both PBS and HTCondor, then thereâs little or no additional admin setup required. Otherwise, youâll need something like BOSCO or HTCondor-CE to handle the jump from HTCondor submit machine to PBS submit machine.
 
You should also consider options where PBS execute nodes can become temporary members of the HTCondor pool. That way, users donât need to decide where their jobs will run. One way to do this is to modify the PBS setup to start HTCondor (configured as an execute node) when the machine is idle and stop HTCondor when it wants to start a PBS job. 
 
Another way to do this is something called GlideIn, where you submit jobs to PBS that are HTCondor daemons (configured as an execute node).
 
Let me know if you want more information on any of these options.
 
Thanks and regards,
Jaime Frey
UW-Madison HTCondor Project
 
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/