[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] limit on input file transfers ????

--On 30 March 2005 12:22 -0600 Zachary Miller <zmiller@xxxxxxxxxxx> wrote:

Have either of you tried this under 6.7? The file transfer in 6.7 should handle this.

i'm sure it would, it's just a matter of efficiency. sending over an entire BLAST db for one condor job does not yield very good througput.

No but if you can run a lot of searches in a single condor job over say several hours then the startup time becomes less of an overhead.

alain's use of rsync, combined with extracting only the useful data from
the blast output is quite nice.  i plan on moving our BLAST framework
here at the UW to something similar.

But there's no rysnc for windows AFAIK. There is also the problem of
permissions. The db files need to be writable for the updates to work
and thus need to put in the user writable area. This is deliberately volatile
so that junk doesn't accumulate in it. Also having relatively large amounts
of disk tied up with condor on a semi-permanent basis is unlikely to
be politically popular.

I'll try 6.7



cheers, -zach

Condor-users mailing list