Re: [Condor-users] condor_history slow

[Jaime Frey <jfrey@xxxxxxxxxxx>]

On Mar 10, 2006, at 1:57 PM, Marco Mambelli wrote:

> Sorry,
> that was the binary, I copied the wrong line the log is 600MB (600K  
> would
> have been not even big...)
> condor_hi 21351 root    3r   REG    8,1 591370531  9224265 /opt/ 
> condor-6.7.9/local.tier2-02/spool/history
>
> schedd at the moment is quite free (~10 jobs running), but probably  
> it is
> the time that takes with a history file around 600MB.
> I wanted just to have an idea of times experienced by other people.

That sounds typical. On one of our systems, condor_history took 50  
seconds to search a 200MB history file.

Whenever you run condor_history, the schedd reads the entire history  
file to find the job ads. We've batted around ideas for creating an  
index for simple queries (specific job ids), but haven't implemented  
anything.

In 6.7.17, you can rotate the history file. You lose information  
about older jobs, but queries will be quicker. As you  mentioned,  
quill is also an option for faster queries.

+--------------------------------+-----------------------------------+
|           Jaime Frey           | I used to be a heavy gambler.     |
|       jfrey@xxxxxxxxxxx        | But now I just make mental bets.  |
| http://www.cs.wisc.edu/~jfrey/ | That's how I lost my mind.        |
+--------------------------------+-----------------------------------+