[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] BLAST



On Monday 21 December 2009 15:15:23 Thomas, Dallas wrote:
> Greetings,
>
>   I have a condor pool configured such that I have the CM on one Linux
> box, another Linux Box is my submit machine and the execute nodes are
> all windows boxes (not my preferred option - but work with what I got).
> Anyways I have a ntfs share that I have mapped on all boxes that
> contains the Global Config, Local Dirs for the Linux boxes and the BLAST
> databases.
>
>   Do I need to install Blast on all of the execute nodes for this to
> work properly?  Does anyone know?

We ended up scp'ing the databases to all execute nodes (for i in 
`condor_status -f '%s\n' machine | sort | uniq) in a script that fires 
condor_submit: network i/o on DB files was killing us.

On linux NR is mmaped to ram in 2GB chunks, if your slot doesn't have that 
much and machine goes thrashing, it'll take forever to finish. By the time I 
figured that out we've decommissioned the last of low-ram boxes here, so I 
never wrote a requirements expression to exclude low-ram nodes. Shouldn't be 
too hard.

We do have the executable on nfs share, there doesn't seem to be much i/o 
contention there. There's a bit more on input and output directories, but not 
enough to move them off nfs.

That's on linux/nfs, no idea how it works on windows w/ ntfs shares.

It's a function of database size and the number of execute slots: with small 
database and a few slots you should be ok with shared directories and low 
ram. If it crawls to a near-halt, watch i/o load on the fileserver and swap 
usage on execute nodes.

(The fastest way is to BLAST everything against everything in one run on a 4x4 
(or 4x6) machine with plenty of ram.)

Dima
-- 
Dimitri Maziuk
Programmer/sysadmin
BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu