Re: [Condor-users] Transfer of output files very slow

[Matthew Farrellee <matt@xxxxxxxxxx>]

user790 user790 wrote:
> I am running Condor on two Windows XP computers. My program is an
> uninstrumented binary running in a vanilla universe. During its execution it
> writes a large (~300) number of small (~1 to 100 kB) files. Now after the
> program has been executed, the files that has been written locally on disk
> are transfered back to the submitter very slowly. It seems it is due to the
> number of files rather than the amount of data, because if I concatenate all
> the files into a single uncompressed archive, it is transferred much faster.
>
> Any idea what is going on and how I could avoid the extra compression step?
>
> Thanks,
>
> P.

On Windows, Condor will do a permissions/ACL check on both sides of the 
file transfer, for each file. Maybe that is an expensive operation in 
your environment.

(BTW, what version of Condor? What's "very slow"?)

Best,


matt