[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [HTCondor-users] Job threads getting held.



Hi!

Bellow are the results of condor_q -better-analyse ClusterID.JobID, and yes, the reasons are failure to transfer and write files from the node to the central manager, to "C:\condor\spool\[ClusterID]\[JobID]\cluster[ClusterID].proc[JobID].subproc0.tmp\

I don't know what could be causing this...

The Job does have an error log file, but it has zero bytes (it's empty).

The nodes where the held jobs where running are completly random. Sometimes it's 4, 9, 11, ...

Thanks for your help!
Alexandre


Here's the output of condor_q -better-analyse ClusterID.JobID:

-----------------------------------------------------------------------------------------------------
C:\Users\Administrator>condor_q -better-analyze 8.255


-- Submitter: condor : <10.2.0.70:49200> : condor
---
008.255:Â Request is held.

Hold reason: Error from slot12@cluster11: STARTER at 10.2.0.61 failed to send file(s) to <10.2.0.70:62822>; SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\8\255\cluster8.proc255.subproc0.tmp\_condor_stdout: (errno 2) No such file or directory

User priority for julieta@xxxxxxxxxx is not available, attempting to analyze without it.
---
008.255: Run analysis summary. Of 264 machines,
ÂÂÂÂÂ 0 are rejected by your job's requirements
ÂÂÂÂÂ 0 reject your job because of their own requirements
ÂÂÂ 170 match and are already running your jobs
ÂÂÂÂÂ 0 match but are serving other users
ÂÂÂÂ 94 are available to run your job
ÂÂÂÂÂÂÂ Last successful match: Fri Oct 03 19:28:09 2014

The Requirements _expression_ for your job is:

ÂÂÂ ( TARGET.Arch == "X86_64" ) && ( TARGET.OpSys == "WINDOWS" ) &&
ÂÂÂ ( TARGET.Disk >= RequestDisk ) && ( TARGET.Memory >= RequestMemory ) &&
ÂÂÂ ( TARGET.HasFileTransfer )

Your job defines the following attributes:

ÂÂÂ DiskUsage = 7500
ÂÂÂ ImageSize = 35000
ÂÂÂ MemoryUsage = 10
ÂÂÂ RequestDisk = 7500
ÂÂÂ RequestMemory = 10
ÂÂÂ ResidentSetSize = 10000

The Requirements _expression_ for your job reduces to these conditions:

ÂÂÂÂÂÂÂÂ Slots
Step Matched Condition
-----Â --------Â ---------
[0]ÂÂÂÂÂÂÂÂ 264Â TARGET.Arch == "X86_64"
[1]ÂÂÂÂÂÂÂÂ 264Â TARGET.OpSys == "WINDOWS"
[3]ÂÂÂÂÂÂÂÂ 264Â TARGET.Disk >= RequestDisk
[5]ÂÂÂÂÂÂÂÂ 264Â TARGET.Memory >= RequestMemory
[7]ÂÂÂÂÂÂÂÂ 264Â TARGET.HasFileTransfer

Suggestions:

ÂÂÂ ConditionÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ Machines MatchedÂÂÂ Suggestion
ÂÂÂ ---------ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ ----------------ÂÂÂ ----------
1ÂÂ ( TARGET.Arch == "X86_64" )ÂÂÂÂÂÂ 264
2ÂÂ ( TARGET.OpSys == "WINDOWS" )ÂÂÂÂ 264
3ÂÂ ( TARGET.Disk >= 7500 )ÂÂÂÂÂÂÂÂÂÂ 264
4ÂÂ ( TARGET.Memory >= 10 )ÂÂÂÂÂÂÂÂÂÂ 264
5ÂÂ ( TARGET.HasFileTransfer )ÂÂÂÂÂÂÂ 264



-----------------------------------------------------------------------------------------------------
Â


C:\Users\Administrator>condor_q -better-analyze 8.249


-- Submitter: condor : <10.2.0.70:49200> : condor
---
008.249:Â Request is held.

Hold reason: Error from slot21@cluster04: STARTER at 10.2.0.54 failed to send file(s) to <10.2.0.70:62937>; SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\8\249\cluster8.proc249.subproc0.tmp\_condor_stdout: (errno 2) No such file or directory

User priority for julieta@xxxxxxxxxx is not available, attempting to analyze without it.
---
008.249: Run analysis summary. Of 264 machines,
ÂÂÂÂÂ 0 are rejected by your job's requirements
ÂÂÂÂÂ 0 reject your job because of their own requirements
ÂÂÂ 156 match and are already running your jobs
ÂÂÂÂÂ 0 match but are serving other users
ÂÂÂ 108 are available to run your job
ÂÂÂÂÂÂÂ Last successful match: Fri Oct 03 19:28:10 2014

The Requirements _expression_ for your job is:

ÂÂÂ ( TARGET.Arch == "X86_64" ) && ( TARGET.OpSys == "WINDOWS" ) &&
ÂÂÂ ( TARGET.Disk >= RequestDisk ) && ( TARGET.Memory >= RequestMemory ) &&
ÂÂÂ ( TARGET.HasFileTransfer )

Your job defines the following attributes:

ÂÂÂ DiskUsage = 7500
ÂÂÂ ImageSize = 35000
ÂÂÂ MemoryUsage = 13
ÂÂÂ RequestDisk = 7500
ÂÂÂ RequestMemory = 13
ÂÂÂ ResidentSetSize = 12500

The Requirements _expression_ for your job reduces to these conditions:

ÂÂÂÂÂÂÂÂ Slots
Step Matched Condition
-----Â --------Â ---------
[0]ÂÂÂÂÂÂÂÂ 264Â TARGET.Arch == "X86_64"
[1]ÂÂÂÂÂÂÂÂ 264Â TARGET.OpSys == "WINDOWS"
[3]ÂÂÂÂÂÂÂÂ 264Â TARGET.Disk >= RequestDisk
[5]ÂÂÂÂÂÂÂÂ 264Â TARGET.Memory >= RequestMemory
[7]ÂÂÂÂÂÂÂÂ 264Â TARGET.HasFileTransfer

Suggestions:

ÂÂÂ ConditionÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ Machines MatchedÂÂÂ Suggestion
ÂÂÂ ---------ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ ----------------ÂÂÂ ----------
1ÂÂ ( TARGET.Arch == "X86_64" )ÂÂÂÂÂÂ 264
2ÂÂ ( TARGET.OpSys == "WINDOWS" )ÂÂÂÂ 264
3ÂÂ ( TARGET.Disk >= 7500 )ÂÂÂÂÂÂÂÂÂÂ 264
4ÂÂ ( TARGET.Memory >= 13 )ÂÂÂÂÂÂÂÂÂÂ 264
5ÂÂ ( TARGET.HasFileTransfer )ÂÂÂÂÂÂÂ 264
-----------------------------------------------------------------------------------------------------
Â


sexta-feira, 3 de Outubro de 2014 20:05
Hi,
ÂIf you run a:
 condor_q -better-analyze ClusterId.JobId Of the Hold job.
Âwhat reasons are shown?
ÂThe task have an Error output?
ÂThe Held task are always in the same nodes?

Bye.




--
Edier Alberto Zapata HernÃndez
Ingeniero de Sistemas
Universidad de Valle
_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/



sexta-feira, 3 de Outubro de 2014 19:27
Hi all,

Iâm having some troubles getting HTCondor to work.

When a user submits a job, some of the threads get held. This is random... For example, in a job with 254 processes, 54 get held.

Has this happened to anyone? Could someone help me?

The central manager and the nodes are all Windows Server 2008 R3 (x64).

Bellow are the logs.

Thanks!!

Alexandre


Here is the related output on SchedLog.log:

[â]
10/03/14 11:49:30 (pid:3420) Starting add_shadow_birthdate(6.237)
10/03/14 11:49:30 (pid:3420) Started shadow for job 6.237 on slot22@cluster09 <10.2.0.59:49163> for julieta, (shadow pid = 8692)
[â]
10/03/14 11:50:25 (pid:3420) Sent vacate command to <10.2.0.59:49163> for job 6.237
[â]
10/03/14 11:50:28 (pid:3420) Shadow pid 8692 for job 6.237 exited with status 112
10/03/14 11:50:28 (pid:3420) Putting job 6.237 on hold
[â]
10/03/14 17:04:37 (pid:3420) OwnerCheck(Administrator) failed in SetAttribute for job 6.237
[â]

And here is the related output on ShadowLog.log:

10/03/14 11:49:30 Initializing a VANILLA shadow for job 6.237
[â]
10/03/14 11:49:32 (6.237) (8692): Request to run on slot22@cluster09 <10.2.0.59:49163> was ACCEPTED
[â]
10/03/14 11:50:25 (6.237) (8692): perm: OpenProcessToken failed: 5
[â]
10/03/14 11:50:25 (6.237) (8692): perm::set_acls(C:\condor\spool\6\237\cluster6.proc237.subproc0): Unable to set file ACL(err=0).
[â]
10/03/14 11:50:25 (6.237) (8692): perm: OpenProcessToken failed: 5
[â]
10/03/14 11:50:25 (6.237) (8692): perm: SetNamedSecurityInfo(C:\condor\spool\6\237\cluster6.proc237.subproc0) failed (err=5)
[â]
10/03/14 11:50:25 (6.237) (8692): (6.237) Failed to chown C:\condor\spool\6\237\cluster6.proc237.subproc0 from to 8256944\8256608.
[â]
10/03/14 11:50:27 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\fort.16, errno = 2: No such file or directory.
10/03/14 11:50:27 (6.237) (8692): get_file(): consumed 16 bytes of file transmission
10/03/14 11:50:27 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\fort.16: (errno 2) No such file or directory
10/03/14 11:50:27 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\GEOME, errno = 2: No such file or directory.
10/03/14 11:50:27 (6.237) (8692): get_file(): consumed 0 bytes of file transmission
10/03/14 11:50:27 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\GEOME: (errno 2) No such file or directory
10/03/14 11:50:27 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\geome1, errno = 2: No such file or directory.
[â]
10/03/14 11:50:27 (6.237) (8692): get_file(): consumed 265438 bytes of file transmission
[â]
10/03/14 11:50:27 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\geome1: (errno 2) No such file or directory
[â]
10/03/14 11:50:27 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\h100238, errno = 2: No such file or directory.
10/03/14 11:50:27 (6.237) (8692): get_file(): consumed 0 bytes of file transmission
[â]
10/03/14 11:50:27 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\h100238: (errno 2) No such file or directory
[â]
10/03/14 11:50:27 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\i100238, errno = 2: No such file or directory.
10/03/14 11:50:27 (6.237) (8692): get_file(): consumed 0 bytes of file transmission
10/03/14 11:50:27 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\i100238: (errno 2) No such file or directory
10/03/14 11:50:27 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\lado12, errno = 2: No such file or directory.
[â]
10/03/14 11:50:27 (6.237) (8692): get_file(): consumed 1061752 bytes of file transmission
10/03/14 11:50:27 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\lado12: (errno 2) No such file or directory
[â]
10/03/14 11:50:27 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\ovalp1, errno = 2: No such file or directory.
[â]
10/03/14 11:50:27 (6.237) (8692): get_file(): consumed 265438 bytes of file transmission
[â]
10/03/14 11:50:27 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\ovalp1: (errno 2) No such file or directory
[â]
10/03/14 11:50:27 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\ovalp2, errno = 2: No such file or directory.
[â]
10/03/14 11:50:27 (6.237) (8692): get_file(): consumed 265438 bytes of file transmission
[â]
10/03/14 11:50:27 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\ovalp2: (errno 2) No such file or directory
10/03/14 11:50:27 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\ovalpm, errno = 2: No such file or directory.
[â]
10/03/14 11:50:27 (6.237) (8692): get_file(): consumed 265438 bytes of file transmission
[â]
10/03/14 11:50:27 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\ovalpm: (errno 2) No such file or directory
[â]
10/03/14 11:50:27 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\p100238, errno = 2: No such file or directory.
[â]
10/03/14 11:50:27 (6.237) (8692): get_file(): consumed 0 bytes of file transmission
[â]
10/03/14 11:50:27 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\p100238: (errno 2) No such file or directory
[â]
10/03/14 11:50:27 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\pontos, errno = 2: No such file or directory.
10/03/14 11:50:27 (6.237) (8692): get_file(): consumed 0 bytes of file transmission
[â]
10/03/14 11:50:27 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\pontos: (errno 2) No such file or directory
10/03/14 11:50:27 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\RECEIVER, errno = 2: No such file or directory.
[â]
10/03/14 11:50:27 (6.237) (8692): get_file(): consumed 532350 bytes of file transmission
[â]
10/03/14 11:50:27 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\RECEIVER: (errno 2) No such file or directory
[â]
10/03/14 11:50:27 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\recfreq_amp, errno = 2: No such file or directory.
10/03/14 11:50:27 (6.237) (8692): get_file(): consumed 0 bytes of file transmission
10/03/14 11:50:27 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\recfreq_amp: (errno 2) No such file or directory

10/03/14 11:50:27 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\recfreq_ri, errno = 2: No such file or directory.
[â]
10/03/14 11:50:27 (6.237) (8692): get_file(): consumed 0 bytes of file transmission
[â]
10/03/14 11:50:27 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\recfreq_ri: (errno 2) No such file or directory
[â]
10/03/14 11:50:27 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\rec_horizontal, errno = 2: No such file or directory.
[â]
10/03/14 11:50:27 (6.237) (8692): get_file(): consumed 430828 bytes of file transmission
[â]
10/03/14 11:50:27 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\rec_horizontal: (errno 2) No such file or directory
[â]
10/03/14 11:50:28 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\rec_perpend, errno = 2: No such file or directory.
10/03/14 11:50:28 (6.237) (8692): get_file(): consumed 194176 bytes of file transmission
10/03/14 11:50:28 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\rec_perpend: (errno 2) No such file or directory
10/03/14 11:50:28 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\rec_vert1, errno = 2: No such file or directory.
10/03/14 11:50:28 (6.237) (8692): get_file(): consumed 162874 bytes of file transmission
10/03/14 11:50:28 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\rec_vert1: (errno 2) No such file or directory
10/03/14 11:50:28 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\res1_absorcao, errno = 2: No such file or directory.
10/03/14 11:50:28 (6.237) (8692): get_file(): consumed 0 bytes of file transmission
10/03/14 11:50:28 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\res1_absorcao: (errno 2) No such file or directory
10/03/14 11:50:28 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\res2_absorcao, errno = 2: No such file or directory.
10/03/14 11:50:28 (6.237) (8692): get_file(): consumed 0 bytes of file transmission
10/03/14 11:50:28 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\res2_absorcao: (errno 2) No such file or directory
10/03/14 11:50:28 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\RESULT, errno = 2: No such file or directory.
[â]
10/03/14 11:50:28 (6.237) (8692): get_file(): consumed 841921 bytes of file transmission
10/03/14 11:50:28 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\RESULT: (errno 2) No such file or directory
[â]
10/03/14 11:50:28 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\s100238, errno = 2: No such file or directory.
10/03/14 11:50:28 (6.237) (8692): get_file(): consumed 0 bytes of file transmission
10/03/14 11:50:28 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\s100238: (errno 2) No such file or directory
[â]
10/03/14 11:50:28 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\v100238, errno = 2: No such file or directory.
10/03/14 11:50:28 (6.237) (8692): get_file(): consumed 0 bytes of file transmission
10/03/14 11:50:28 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\v100238: (errno 2) No such file or directory
10/03/14 11:50:28 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\_condor_stderr, errno = 2: No such file or directory.
10/03/14 11:50:28 (6.237) (8692): get_file(): consumed 783 bytes of file transmission
10/03/14 11:50:28 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\_condor_stderr: (errno 2) No such file or directory
10/03/14 11:50:28 (6.237) (8692): get_file(): Failed to open file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\_condor_stdout, errno = 2: No such file or directory.
10/03/14 11:50:28 (6.237) (8692): get_file(): consumed 4544 bytes of file transmission
10/03/14 11:50:28 (6.237) (8692): DoDownload: consuming rest of transfer and failing after encountering the following error: SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\_condor_stdout: (errno 2) No such file or directory
[â]
10/03/14 11:50:28 (6.237) (8692): Mock terminating job 6.237: exited_by_signal=FALSE, exit_code=8 OR exit_signal=0, core_dumped=FALSE, exit_reason="Exited normally"
[â]
10/03/14 11:50:28 (6.237) (8692): Job 6.237 going into Hold state (code 12,2): Error from slot22@cluster09: STARTER at 10.2.0.59 failed to send file(s) to <10.2.0.70:50914>; SHADOW at 10.2.0.70 failed to write to file C:\condor\spool\6\237\cluster6.proc237.subproc0.tmp\_condor_stdout: (errno 2) No such file or directory
[â]
10/03/14 11:50:28 (6.237) (8692): **** condor_shadow (condor_SHADOW) pid 8692 EXITING WITH STATUS 112
[â]