[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Condor-users] The /var/lib/condor/execute folder 1, 000, 000 question? :-)



Hi Sassy:

I think your question assumes that there is an underlying shared
filesystem (e.g. your home directory is mounted on both the schedd
machine and the startd machine). If this is the case, and
FILESYSTEM_DOMAIN in your config file is set correctly (the values
should be equivalent on both your submit and execute machines if they
share this filesystem), then your executable should run in the submit
directory on the execute machine, provided your submit file has an
appropriately defined should_transfer_files line.

If you want to ensure that your job only runs on machines where the
FILESYSTEM_DOMAIN value match, and thus your home directory is mounted,
make sure you specify
should_transfer_files = ALWAYS
in your submit file.

In this case, a directory will still get created in EXECUTE or
SLOT_<N>_EXECUTE which your job can use (for temporary local storage,
for instance), but your executable should start running in the submit
directory.

I believe the default value of should_transfer_files is IF_NEEDED
(although I haven't looked this up in a while), which means that your
job will start in the EXECUTE or SLOT_<N>_EXECUTE directory on machines
where the FILESYSTEM_DOMAINs don't match and will start in the submit
directory on machines where the FILESYSTEM_DOMAINs do match.

Does this answer your question? Perhaps I am missing something crucial.

Cheers,
David

On Thu, 2011-07-21 at 01:11 +0300, Sassy Natan wrote:
> any idea?
> 
> On Wed, Jul 20, 2011 at 3:59 PM, Sassy Natan <sassyn@xxxxxxxxx> wrote:
> 
> > Well, the initialdir doesn't make the process run in specified directory
> > but it does create the log files in it.
> >
> > what i'm interesting is to send a job to the q, and have it run in my local
> > home directory, where I was when submitting the job. so it is like i run the
> > command locally without using condor.
> >
> > so for example if i use gcc, the command with condor will run in the ~
> > home, and not in /var/lib/condor/execute/%jobid% folder.
> > I saw there is a remote_initialdir option and CONDOR_SCRATCH_DIR option,
> > but i'm not sure this will help me.
> >
> > sassy
> >
> >
> >
> > On Wed, Jul 20, 2011 at 3:48 PM, Matthew Farrellee <matt@xxxxxxxxxx>wrote:
> >
> >> Check the condor_submit manual page for initialdir.
> >>
> >> Best,
> >>
> >>
> >> matt
> >>
> >>
> >> On 07/19/2011 05:22 PM, Sassy Natan wrote:
> >>
> >>> Thank,
> >>>
> >>> But still, changing the path is easy. I would like it to be
> >>> a dynamic path, pointing to the working directory the user submitted the
> >>> job from. Which in my case is always happening inside the user home
> >>> directory.
> >>>
> >>> This /home is NFS volume on a local filers, running CephFS as there FS
> >>> (under RDBD volume).
> >>>
> >>> The SLOT<N>_EXECUTE is also not wise, what if i use over and over the
> >>> same slot no 4? no other slot are being used?
> >>>
> >>> Sassy
> >>>
> >>>
> >>> On Wed, Jul 20, 2011 at 12:09 AM, Erik Erlandson <eje@xxxxxxxxxx
> >>> <mailto:eje@xxxxxxxxxx>> wrote:
> >>>
> >>>    On Tue, 2011-07-19 at 23:57 +0300, Sassy Natan wrote:
> >>>
> >>>     > Is it possible to change the directory where the job processes is
> >>>     > truly running? AKA /var/lib/condor/execute?
> >>>
> >>>    The EXECUTE (or SLOT<N>_EXECUTE) configuration variable can control
> >>> the
> >>>    directory jobs execute in:
> >>>    http://www.cs.wisc.edu/condor/**manual/v7.6/3_3Configuration.**
> >>> html#15493<http://www.cs.wisc.edu/condor/manual/v7.6/3_3Configuration.html#15493>
> >>>
> >>
> >>
> >
> _______________________________________________
> Condor-users mailing list
> To unsubscribe, send a message to condor-users-request@xxxxxxxxxxx with a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/condor-users
> 
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/condor-users/