[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [HTCondor-users] Issues Running BLAST through HTCondor
- Date: Tue, 08 Mar 2022 21:10:26 -0600
- From: dmitri.maziuk@xxxxxxxxx
- Subject: Re: [HTCondor-users] Issues Running BLAST through HTCondor
On 2022-03-08 6:44 PM, sameh.abedin@xxxxxxxxx wrote:
I have 3 hosts that can execute jobs. 1 host has 16 slots, and the other
2 have 8 slots. So, 32 slots total.
How much RAM?
The best way to run blast is to put the databases on a RAM disk and run
multi-threaded on all available cores, with all of your query sequences
in the single query file ("all against all"). I.e. not really the way
condor does it.
I would try setting up whole-machine slots and only running one blast
instance per machine. It shouldn't be too hard to give it the correct
"numThreads" argument but you could start with the lowest common
denominator: 8. Your search DB seems small enough that RAM shouldn't
matter, but you really need enough RAM to hold it -- or 2GB, whichever's
smaller -- *per slot* just for blast.
That said, we only ran protein blast and when they switched to the new
and improved "blast+" it started taking 10 times longer which
effectively killed our pipeline. So I'm not sure what's happening with
your setup, exactly. Not enough RAM would be my first suspect if you
were searching nr or nt, but with your database I don't know.