[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Problem with output & error paths



Hello TJ, I just wanted to confirm that my preliminary tests show this problem fixed in v8.6.13.

Thanks a lot!

 

Óscar

 

 

De: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> En nombre de John M Knoeller
Enviado el: miércoles, 29 de agosto de 2018 18:25
Para: HTCondor-Users Mail List <htcondor-users@xxxxxxxxxxx>; condor-users@xxxxxxxxxxx
Asunto: Re: [HTCondor-users] Problem with output & error paths

 

Yes, this is a bug that we just discovered ourselves.  When the output and error files are full paths, the filename rewriting code kicks in, and that code treats a \ as an escape character so that it can support = or ; in a filename.   This is a bug, and we will be fixing it in an upcoming release.

 

As a workaround I think you can make it behave the same on the submitting host as on the remote machines by putting

 

should_transfer_files = YES

 

in your submit files.   This way you will *always* need to double the \. 

 

In most cases on Windows, you can you / instead of \.   so your paths could be

 

\\\\server\\share/path/to/file

 

or perhaps

 

X:/path/to/file

 

If you set up a drive letter for the share.

 

You might be able to get away with using / everywhere. like this //server/share/path/to/file

 

I don’t know if that will work or not.

 

The other thing you could do to work around this is to set initialdir in your submit files, like this

 

should_transfer_files = YES

initialdir = \\server\share\path\to

output = file.out

error = file.err

log = file.log

 

-tj

 

From: HTCondor-users <htcondor-users-bounces@xxxxxxxxxxx> On Behalf Of Laborda Sanchez, Oscar (Volkswagen Group Services)
Sent: Wednesday, August 29, 2018 5:02 AM
To: condor-users@xxxxxxxxxxx
Subject: [HTCondor-users] Problem with output & error paths

 

Hello,

 

I am using HTCondor 8.6.11 on Windows submitting a job that has output, error and log files directed to a windows share.

When I write the paths in a standard way, i.e:

output = \\smb-server\path\to\file.out

error = \\smb-server\path\to\file.err

log = \\smb-server\path\to\file.log

 

The job works fine only when executed in the subitting machine (it also happens to be the master), but when it is executed in another machine, it generates file.out file.err with size 0 and then tries to open file paths:

\smb-serverpathtofile.out

\smb-serverpathtofile.err

 

So either I get those files in the root directory of the submitter when the user has rights to create them, or an error message in the log file:

 

007 (012.000.000) 08/28 10:15:58 Shadow exception!

                Error from slot1_1@fqdn-executing-machine: STARTER at ip-executing-machine failed to send file(s) to <ip-submitter-machine:9618>; SHADOW at ip-submitter-machine failed to write to file \smb-serverpathtofile.out: (errno 13) Permission denied

                572  -  Run Bytes Sent By Job

                0  -  Run Bytes Received By Job

...

012 (012.000.000) 08/28 10:15:58 Job was held.

                Error from slot1_1@fqdn-executing-machine: STARTER at ip-executing-machine failed to send file(s) to <ip-submitter-machine:9618>; SHADOW at ip-submitter-machine failed to write to file \smb-serverpathtofile.out: (errno 13) Permission denied

                Code 12 Subcode 13

 

So the problem seems to be that I need to escape the backslashes. I have then tried to write the paths like this, doubling the backslashes:

 

output = \\\\smb-server\\path\\to\\file.out

error = \\\\smb-server\\path\\to\\file.err

log = \\\\smb-server\\path\\to\\file.log

 

This way the job works for remote execution machines, but not in the submitting host. In the submitter, again the .out and .err files get created with size 0, but then I get this in the .log:

 

007 (027.000.000) 08/28 12:19:24 Shadow exception!

                Error from slot1_1@fqdn-submitting-machine: Failed to open '\\\\smb-server\\path\\to\\file.out' as standard output: No such file or directory (errno 2)

                0  -  Run Bytes Sent By Job

                0  -  Run Bytes Received By Job

...

012 (027.000.000) 08/28 12:19:25 Job was held.

                Error from slot1_1@fqdn-submitting-machine: Failed to open '\\\\smb-server\\path\\to\\file.out' as standard output: No such file or directory (errno 2)

                Code 7 Subcode 2

 

Looks like the submitter does not like the escaping technique...

The rest of the options I am using are these:

universe = vanilla

run_as_owner = true

transfer_executable = false

 

Note that the .log file is always created and perfectly written to using either method of writing the paths.

 

Is there a way to write the paths so the job works both in the submitter and remote machines?

 

Thanks a lot!

Oscar