[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Issues Running BLAST through HTCondor

On 2022-03-08 6:44 PM, sameh.abedin@xxxxxxxxx wrote:
I have 3 hosts that can execute jobs. 1 host has 16 slots, and the other 2 have 8 slots. So, 32 slots total.

How much RAM?

The best way to run blast is to put the databases on a RAM disk and run multi-threaded on all available cores, with all of your query sequences in the single query file ("all against all"). I.e. not really the way condor does it.

I would try setting up whole-machine slots and only running one blast instance per machine. It shouldn't be too hard to give it the correct "numThreads" argument but you could start with the lowest common denominator: 8. Your search DB seems small enough that RAM shouldn't matter, but you really need enough RAM to hold it -- or 2GB, whichever's smaller -- *per slot* just for blast.

That said, we only ran protein blast and when they switched to the new and improved "blast+" it started taking 10 times longer which effectively killed our pipeline. So I'm not sure what's happening with your setup, exactly. Not enough RAM would be my first suspect if you were searching nr or nt, but with your database I don't know.