[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Transfer of output files very slow

user790 user790 wrote:
I am running Condor on two Windows XP computers. My program is an
uninstrumented binary running in a vanilla universe. During its execution it
writes a large (~300) number of small (~1 to 100 kB) files. Now after the
program has been executed, the files that has been written locally on disk
are transfered back to the submitter very slowly. It seems it is due to the
number of files rather than the amount of data, because if I concatenate all
the files into a single uncompressed archive, it is transferred much faster.

Any idea what is going on and how I could avoid the extra compression step?



On Windows, Condor will do a permissions/ACL check on both sides of the file transfer, for each file. Maybe that is an expensive operation in your environment.

(BTW, what version of Condor? What's "very slow"?)