Re: [HTCondor-users] condor_history JSON malformed with multiple files

[Jaime Frey <jfrey@xxxxxxxxxxx>]

On Oct 5, 2017, at 8:55 AM, Fischer, Max (SCC) <max.fischer@xxxxxxx> wrote:

I am seeing some spurious errors when dumping condor_history as json. After about 4000/26000 jobs, there is a list close/open inserted [1].
So, the JSON is formatted as
..., {...}][, {...}, ...
when it should be
..., {...}, {...}, ...
with {...} as individual job data mappings.
The attributes do not seem to matter, even if I have all, 1 or no attribute, it breaks at the same number of jobs [2].

It seems the error is that condor_history adds the list open/close for every file read [3], and we have multiple history files. If files are read by the schedd via `-name $(hostname)`, the problem does not occur [4].

Thatâs definitely not right. The problem appears to be exactly as you describe, where ads from each history file are printed in their own list, with nothing between the list begin/end delimiters. The same problem happens with the -xml option. We will work on a fix for the next release.

Thank you for the detailed report.

Thanks and regards,

Jaime Frey

UW-Madison HTCondor Project