[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[HTCondor-users] adstash / grafana question: grouping by Cmd basename



Hello,

I would like to build a dashboard in which job history metrics gathered by adstash and pumped to elasticsesarch are grouped by executable *basename*.

The motivation is we have users (myself included) who run jobs from e.g. shell scripts copied to an execute directory but which are fundamentally the same process - I would like to monitor such jobs for failures and general performance, broken down by the IGWN/OSG site at which they ran.

My plan was to build a variable with:

{"find": "terms", "field": "Cmd"}

and strip out the path with a regex, leaving the basename - that gives me a drop-down menu of executable basenames I'd like to use in the panel query.

My immediate problem seemed to be that Cmd is not an indexed field, so it looks like I cannot do any kind of straightforward searching on that.

Instead, I figured maybe I can just abandon using a variable in a query and simply group-by the Cmd field and then use a rename-by-regex transform to give me the basenames. That at least gives me a panel where I can select individual traces to give basically the same result.

One problem: I just get a ton of traces with identical names (corresponding to the now absent paths).

Anyone know if there's a further grafana transform I can use to group those identically-named results together?

Or maybe the better thing to do is just add an attribute, either through a job transform or (maybe) elasticsearch ingest node pipeline, that has the desired information so it's there in the db in the first place?

Hopefully this isn't too obscure / into the weeds to be meaningful..
Thanks

--
James Alexander Clark
LIGO Laboratory
California Institute of Technology
email:  james.clark@xxxxxxxx
Tel. (cell):  413-230-1412