Re: [HTCondor-users] Sharing data across nodes

[dmitri maziuk <dmitri.maziuk@xxxxxxxxx>]

On 2022-05-17 2:00 PM, Michael Thomas wrote:
> Hi Krishna,
>
> A distributed filesystem such as ceph (www.ceph.io) or hdfs 
> (https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html) 
> may provide what you need.

I played with ceph long time ago when it was still young. At the time 
there was no options to "pin" a client to data node and no plan to make 
it available to admins (IIRC there was something you could do at compile 
time).

Which means you could set up local ceph storage on a condor node, but 
couldn't make that condor node do all its i/o on its local ceph storage. 
Ceph would spread the i/o over all its storage nodes acording to its 
clever algorithm; it was be better than all condor hosts hitting a 
single storage server (nfs that we wanted to replace), but to get any 
i/o performance you'd need 10Gb fabric.

That may have changed since.

Dima