[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] Transfer of output files very slow

I am running the latest version, condor 7.2.0. Very slow means about 15 minutes for transfering 10MB. I'll indeed dig into security checks to see if I can improve this latency, thanks.
2008/12/31 Matthew Farrellee <matt@xxxxxxxxxx>
user790 user790 wrote:
> I am running Condor on two Windows XP computers. My program is an
> uninstrumented binary running in a vanilla universe. During its execution it
> writes a large (~300) number of small (~1 to 100 kB) files. Now after the
> program has been executed, the files that has been written locally on disk
> are transfered back to the submitter very slowly. It seems it is due to the
> number of files rather than the amount of data, because if I concatenate all
> the files into a single uncompressed archive, it is transferred much faster.
> Any idea what is going on and how I could avoid the extra compression step?
> Thanks,
> P.

On Windows, Condor will do a permissions/ACL check on both sides of the
file transfer, for each file. Maybe that is an expensive operation in
your environment.

(BTW, what version of Condor? What's "very slow"?)


Condor-users mailing list
To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting

The archives can be found at: