I have a question regarding a behavior of condor that is puzzling me. I have Condor on a single standalone Windows machine as both server and client. I am trying to run a job that goes great and seems to produce all the outputs (in C:\execute\dirWXYZ). When I look at my "initialdir" directory (C:\study\output) I have about 80% of the output files in there.
From watching this thing run 6 times in a row, here is what I think is happening:
* My job runs successfully, producing outputs that are 3450 CSV files
* I can tell my job ends because the last part of the script deletes XML files; they are gone from C:\execute\dirWXYZ
* Condor, I think, starts to move these CSVs over to C:\study
* It gets through about 80% of them and apparently has an error
* Condor re-runs the job from the start due to the error
So I am trying to debug this an hoping someone might have seen a similar behavior or know a fix. There are lots of files, but the size comes to only 5-7MB. I found a post online about output files in the 2GB range causing problems, but nothing else obviously relevant.
Thanks in advance,