Re: [HTCondor-users] query transfer history like "condor_history"

[Jason Patton <jpatton@xxxxxxxxxxx>]

Hi Xiaowei,

I wouldn't call this a solution, but here's a hacky way to get "condor_status" to approach what you're looking for. (The "sed" command is needed to get rid of the "***" classad separators in the transfer_history file, then "condor_status -ads -" reads the classads from stdin.)

$ sed -e's/^\*\*\*.*$//' /var/log/condor/transfer_history | condor_status -limit 5 -af:h JobClusterId JobProcId TransferType TransferProtocol TransferSuccess -ads -
JobClusterId JobProcId TransferType TransferProtocol TransferSuccess
14367653     982       download     cedar            true           
14367653     982       download     cedar            true           
14367653     982       download     cedar            true           
14367653     982       download     cedar            true           
14367653     1500      download     cedar            true

Most of the condor_status options should work here for formatting the transfer_history attributes and constraining your queries however you like. A nicer solution might be to parse the transfer_history file using the classad library in the HTCondor Python bindings.

(h/t to Todd Miller for the initial idea to use condor_status!)

Jason Patton

On 10/27/21 8:53 PM, JIANG Xiaowei wrote:

Dear experts,

We are transferring data via htcondor transfer mechanism and need some monitor to check the status of transfer volume and throughput.

Can htcondor, similar with "condor_history", query a transfer_history file and print out the formatted summarized data?

Any suggestion would be appreciated!

Best,

Xiaowei 

_______________________________________________
HTCondor-users mailing list
To unsubscribe, send a message to htcondor-users-request@xxxxxxxxxxx with a
subject: Unsubscribe
You can also unsubscribe by visiting
https://lists.cs.wisc.edu/mailman/listinfo/htcondor-users

The archives can be found at:
https://lists.cs.wisc.edu/archive/htcondor-users/