[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] shared processing folder???



Thanks for your Reply Christoph,

Our HTCondor cluster is just a private LAN with one master node as gateway and a few processing nodes. Under this paradigm, different hosts are no needed.

I vaguely imagined to share a folder in master node (with appropriates permits) and the jobs can consume files from this ubication.

We are attemping completely minimize transfer (through input_transfer_files) on each node. 

This would be possible?

 

Regards

Ale

 

De: HTCondor-users [mailto:htcondor-users-bounces@xxxxxxxxxxx] En nombre de Beyer, Christoph
Enviado el: viernes, 6 de marzo de 2020 11:31
Para: htcondor-users
Asunto: Re: [HTCondor-users] shared processing folder???

 

Hi Ale,

 

do you think of a static directory with a bunch of files, that can be created just as you would on a regular host maybe through a ssh-loop.

 

If it is more about a common base of programs etc. cvmfs comes to mind.

 

If the stuff you host in that dir is more dynamic you could host it on a nfs file server etc.

 

If you want your jos t make sure this 'dir' is present on the workernode you could define an according filesystem domain and request that domain in your job....

 

Best

Christoph

 


--
Christoph Beyer
DESY Hamburg
IT-Department

Notkestr. 85
Building 02b, Room 009
22607 Hamburg

phone:+49-(0)40-8998-2317
mail:
christoph.beyer@xxxxxxx

 


Von: "Alejandro Acuña" <alejandro.acunia@xxxxxxxxxxxxxxxx>
An: "htcondor-users" <
htcondor-users@xxxxxxxxxxx>
Gesendet: Freitag, 6. März 2020 13:38:22
Betreff: [HTCondor-users] shared processing folder???

 

Hi friends.

I need to specify a folder with specific shared files for all jobs processing (like a input_transfer_files). I wonder if exists a kind of make available this folder to all processing nodes and put in there all of necesary files (probably files must be readed or writed too).

Finally, if it’s posible, how is the submit?

 

Thanks!!!

Ale

 


_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to
htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/