pcp
[Top] [All Lists]

Re: RFC: filtered metrics

To: "Frank Ch. Eigler" <fche@xxxxxxxxxx>
Subject: Re: RFC: filtered metrics
From: Nathan Scott <nathans@xxxxxxxxxx>
Date: Tue, 8 Oct 2013 23:07:01 -0400 (EDT)
Cc: pcp@xxxxxxxxxxx
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <y0mwqlpyx0x.fsf@xxxxxxxx>
References: <1717961238.2089908.1380100281235.JavaMail.root@xxxxxxxxxx> <y0mvc1g5tyg.fsf@xxxxxxxx> <1551478303.2549744.1381116552518.JavaMail.root@xxxxxxxxxx> <y0mwqlpyx0x.fsf@xxxxxxxx>
Reply-to: Nathan Scott <nathans@xxxxxxxxxx>
Thread-index: wZ79Gs5pPRrisEXJs8uv5/0SHtP+xg==
Thread-topic: filtered metrics

----- Original Message -----
> 
> nathans wrote:
> 
> >> [...]
> >> Maybe they could be left in files.  Or maybe the pcp clients can earn
> >> an additional option like pmevent, '-x [metric:]filter'.  pmlogger
> >
> > I'm trying to avoid that, as it means new code needed for each client.
> 
> To some extent, OTOH we already have some common command line argument
> parsing/handling in libpcp.
> 

In the pmda library we do, but not libpcp (parsing, anyway) - but, we
should tackle that (other planned little projects need it too).

> 
> > Also, good chance some PMDAs will need filters that are quite complex
> > to specify on the command line (like scripts).
> 
> So instead of passing arguments on the command line, you'd imagine
> passing arguments in a temporary file?  That's not unheard of (like
> DOS's @FILE), but usually thought of as a last resort.
> 

Not sure why temporary?  Thinking system wide filters like "when looking
at per-process metrics, this system is expected to have N million insts,
so force a system wide filter for all access to per-process metrics via
a specified cgroup".

FWIW, derived metrics use config file via environment variable ... which
may be a just-fine model here too.  I like standard, permanent location
options as well for cases where one knows, really, really knows that they
*always* want some filter (or derived metric, or...) to exist.

If I step away from my earlier "don't want to change individual clients"
comment, as I should, then command line options make alot of sense for
specifying filters too.

> >> > [...] Should we mandate use of string metrics always for these
> >> > things?  (the one case we have that used integers could be done as a
> >> > string instead, and its not yet released).  [...]
> >> 
> >> Substring match?
> >> Regular expression?
> >> Arithmetic expression?
> >> General PMIE predicate?
> >
> > *nod* wireshark filter?, on/off?, sql? - infinite other options.  We
> > should ensure all of the above are readily achievable & allow people
> > using them to be creative in coming up with their own filter concepts.
> 
> Back up a bit.  It would help to give a worked-out example or three
> about what these could look like.  Obviously, the more specialize the
> filtering, the more it has to be pushed toward the PMDAs and/or
> clients, but OTOH that makes the PMAPI / pmcd contribution rather
> slim or even zero.

I think it should be (has to be?) up to the PMDA as to how filters will
be applied, and to interpret what makes sense for its domain.  All the
examples I had were specific to the domains they were in (e.g. logfile
filter -> regex) - so I think in terms of interpreting the filters,
PMAPI/pmcd has no contribution to make other than providing transport
and sending the filter string through at the appropriate time(s).

Will ponder more deeply & send out some worked examples down the track.

cheers.

--
Nathan

<Prev in Thread] Current Thread [Next in Thread>