Re: [Condor-users] (no subject)

["Douglas Thain" <dthain@xxxxxx>]

One way to attack these problems is to use Parrot and Chirp,
which allow ordinary programs to easily access files over the network.
It's like NFS except that you don't need any help from the system
admin to get it running.

Here is general info about Parrot and Chirp:

    http://www.cse.nd.edu/~ccl/software/parrot
    http://www.cse.nd.edu/~ccl/software/chirp

And, here is a little writeup that explains how to submit Condor jobs
that do remote dynamic linking over the network using Parrot and Chirp.

    http://www.cse.nd.edu/~ccl/software/howto/remotelink.shtml

There are some limitations to the system (e.g. only 32/64-bit Linux),
but it works for a fair number of applications.

Cheers,
Doug


On Tue, Sep 16, 2008 at 1:37 PM, Pascal Jermini
<pascal+condor@xxxxxxxxxxxxxxxx> wrote:
> Hi,
>
>> I wanted to ask for some advice about executables when you are NOT
>> sharing filesystems. I'm bringing up a condor cluster, using student
>> lab machines, that won't have user accounts on them, nor shared
>> application directories.
>
> We are in a similar set-up for our pool, where there are almost no
> pre-installed software on the compute nodes, and we don't have any access to
> the compute nodes.
>
>> e.g.  bioperl has a great many dependencies, and is a complex install.
>> Do you have to transfer all of the libraries, etc.?
>
> Unfortunately yes. Our users end up transferring all the required libraries.
> We strongly recommend to perform static linking whenever possible, as it is
> easier then to execute on compute nodes where we don't have administrative
> access.
>
> Sometimes we also advise to create a tar.gz (or a .zip) file containing all
> the libraries and executable, and as a first task untar the file on the
> compute node to get the executables and the dependencies...
>
>> What do you transfer to one of the anonymous worker bees?
>> I guess, statically linked stuff would work, but not all applications
>> have that option.
>
> You are right...some applications cannot be statically linked, so we try to
> link statically as much as possible, then the remaining shared libraries get
> transferred to the compute node alongside the executable and input files.
>
> Pascal
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/
>