[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] bittorrent with condor

On 8/18/06, beck lee <beck_jn@xxxxxxxxxxxx> wrote:


Has anyone else used the bittorrent protocol to transfer input files ? If
you have, please tell me some experience include how to do and compare to
the traditional condor transfer protocol, which is better.

The bittorrent protocol is expressly deigned around transmitting a
file to multiple places at once and using 'cross-talk' between nodes
involved once the nodes start to accumulate parts of the data from the
initial small number of seed start points.

If you are transfering from one place, to one other (or indeed
transferring from multple machines to one in all but the most
unrealistic upstream/downstream possibilites) then the bittorrent
protocol is at best pointless and at worst actively degres performance
due to the overhead.

Without knowing some domain specific knowledge about your data
transfer requirements and common use cases to suggest any differnt
transfering the data yourself via bittorrent is likely to be a bad

Transfering yourself by some other method (using again domain specific
knowldge about your network) where your job can drop the data
somewhere itself rather than transferring back to then be passed on
can be very beneficial. For this bittorrent would not be a good fit