Re: [HTCondor-users] move spool and log folders on highspeednetwork drive

[Antonis Sergis <sergis_antonis@xxxxxxxxxxx>]

yes there is full access to the network location by condor and the 
submitting machines (read and write) . The condor service stops and I cannot 
get it restarted as soon as I change the locations of the spool and log 
folders to the network location inside the config file. Through cmd condor 
does not respond to pretty much any commands giving me a "Can't connect to 
local master" error

A.

-----Original Message----- 
From: Andrey Kuznetsov
Sent: Thursday, March 28, 2013 6:59 PM
To: HTCondor-Users Mail List
Subject: Re: [HTCondor-users] move spool and log folders on highspeednetwork 
drive

Does condor have permission to write to location?
Is location accessible by that machine?

You said that if you change log and swap location, condor starts up,
then it must be an access/write issue over the network.

On Thu, Mar 28, 2013 at 11:55 AM, Antonis Sergis
<sergis_antonis@xxxxxxxxxxx> wrote:
> Hello Andrew,
>
> no I am using separate pools with separate network storage locations for
> each log and spool
>
> Antonis
>
> -----Original Message----- From: Andrey Kuznetsov
> Sent: Thursday, March 28, 2013 6:29 PM
>
> To: HTCondor-Users Mail List
> Subject: Re: [HTCondor-users] move spool and log folders on high
> speednetwork drive
>
> Are you trying to set 2 pool machines to use the same log file?
> How do you think 2 machines are supposed to place a lock on the same file?
>
> Though, I don't know how multipool setup works.
>
> On Thu, Mar 28, 2013 at 10:11 AM, Antonis Sergis
> <sergis_antonis@xxxxxxxxxxx> wrote:
> > hello. I am writing to get some more ideas regarding a problem which is
> > becoming rather hard to tackle. I have my machine as a condor submitter
> > and
> > unfortunately we realised that the local disk transfer speeds for the log
> > and spool files is too slow and limits our maximum job number. Replacing
> > the
> > disk with an ssd will bring another problem close which is processor
> > speed.
> > I have hence decided to alter the config file to be able to make the
> > submitting machine exchanging data over our super fast connection and
> > network storage. I had a go trying out different things for the last 
> > days.
> > I
> > got it to momentarily work and the number of jobs I could carry out
> > simultaneously went up to 1200 from 300 which was the earlier limit
> > however
> > then the processor maxed out and cut off taking up more jobs. We are
> > planning to split the administration job to other PCs to get the
> > processing
> > speed required and the max amount of jobs running. I have tried adding 
> > the
> > network location folders for the spool and log pathnames in the
> > configuration file:
> >
> > ######################################################################
> > ##  Daemon-wide settings:
> > ######################################################################
> >
> > ##  Pathnames
> > LOG        = \\PATHNAME\log
> > SPOOL        = \\PATHNAME\spool
> > EXECUTE        = $(LOCAL_DIR)/execute
> > BIN        = $(RELEASE_DIR)/bin
> > LIB        = $(RELEASE_DIR)/lib
> > INCLUDE        = $(RELEASE_DIR)/include
> > SBIN        = $(BIN)
> > LIBEXEC        = $(BIN)
> >
> > However this does not work and the condor service is cut off and I cannot
> > restart it or enquire about it unless I change the config file back to 
> > the
> > initial one (i.e. local log and spool folders). I am running condor on a
> > windows 7 machine. Replacing the memory with an SSD is not an option as
> > the
> > job sizes are quite large and there are no funds to do that on a large
> > scale
> > while the network storage can provide the speed we are after. Any ideas?
> >
> > Cheers
> >
> > Antonis
> >
> > _______________________________________________
> > HTCondor-users mailing list
> > To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with
> > a
> > subject: Unsubscribe
> > You can also unsubscribe by visiting
> > https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
> >
> > The archives can be found at:
> > https://lists.cs.wisc.edu/archive/htcondor-users/
>
>
>
>
> --
> Andrey Kuznetsov <akuznet1@xxxxxxxx>
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with 
> a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/
> _______________________________________________
> HTCondor-users mailing list
> To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with 
> a
> subject: Unsubscribe
> You can also unsubscribe by visiting
> https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users
>
> The archives can be found at:
> https://lists.cs.wisc.edu/archive/htcondor-users/



--
Andrey Kuznetsov <akuznet1@xxxxxxxx>
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/