pcp
[Top] [All Lists]

Re: [pcp] Archive discovery, was Re: __pmDiscoverServices()/pmfind Enha

To: "Frank Ch. Eigler" <fche@xxxxxxxxxx>, Dave Brolley <brolley@xxxxxxxxxx>
Subject: Re: [pcp] Archive discovery, was Re: __pmDiscoverServices()/pmfind Enhancements
From: Nathan Scott <nathans@xxxxxxxxxx>
Date: Thu, 22 May 2014 05:53:23 -0400 (EDT)
Cc: PCP Mailing List <pcp@xxxxxxxxxxx>
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <y0m7g5l2rzq.fsf@xxxxxxxx>
References: <5373AAA7.70004@xxxxxxxxxx> <y0m7g5l2rzq.fsf@xxxxxxxx>
Reply-to: Nathan Scott <nathans@xxxxxxxxxx>
Thread-index: 3KaBUKosylgseAcyjd185Tgds8mTNw==
Thread-topic: Archive discovery, was Re: __pmDiscoverServices()/pmfind Enhancements

----- Original Message -----
> 
> Hi -
> 
> While hacking on pmwebd extensions, it occurred to me that we could
> offer archive discovery capabilities within the pmDiscoverServices()
> function.  This could be handy for interactive users to populate file
> browsers or live web popups, and possibly for tools too.

I like the general idea of finding archives (which the unified context
mode will need), but not via pmDiscoverServices - new API(s) would make
more sense IMO.

pmfind could still make a good front-end (using new APIs in addition to
the existing), but archives are not services - and the needs differ, as
becomes clear from the examples (esp. the filtering ones, below).

> [...] ",basedir=xxx"

I have some security reservations about allowing remote users to poke at
a far away system via pmwebd to determine whether arbitrary files exist
or not (based on return codes or response time) - it would have to be
limited to a set of well-known locations (eg /var/log/{pmlogger,pmmgr}).

And its not clear raw archive paths should be exposed at all via the web
API, a unified context with multiple archive support seems cleaner.

>  - filter for archive-label matching given hostname
>                                  ",loghost=BAR.BAZ"
>  - filter for archive-label covering all given TIME(s)
>                                  ",logtime=TIME,logtime=TIME"
>  - filter for archive metadata including metric
>                                  ",metric=PMNS"

This could be an arbitrarily complex filtering language - sql-like
even, so forcing everything through the already-poorly-specified
"mechanism" parameter in an API for service discovery is not going
to fly - we should start afresh, and make the archive API clean (&
suitable internally for unified context mode too, of course).

>  - limit directory recursion
>                                  ",maxdepth=NN"
>  - limit time
>                                  ",timeout=NN"
> 

With a fixed set of locations, these last examples become unnecessary
(although the streaming concept Dave talked about, via callbacks, I'm
a big fan of - for archives too).

Hmm, above suggests we should've thought of an "options" parameter to
pmDiscoverServices(3).  If we end up needing to revisit the existing
services API/ABI, Dave, we should definitely split "options" out from
"mechanism" in any new variant.  Ah - we haven't added any of that in
yet (the flags-smushed-in-with-mechanism bit) have we? ... hmmm.  We
should probably split that apart right away.

cheers.

--
Nathan

<Prev in Thread] Current Thread [Next in Thread>