pcp
[Top] [All Lists]

Re: [pcp] sosreport and pcp

To: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Subject: Re: [pcp] sosreport and pcp
From: Michele Baldessari <michele@xxxxxxxxxx>
Date: Tue, 6 May 2014 13:15:05 +0100
Cc: pcp@xxxxxxxxxxx
Delivered-to: pcp@xxxxxxxxxxx
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/simple; d=acksyn.org; h= user-agent:in-reply-to:content-disposition:content-type :content-type:mime-version:references:message-id:subject:subject :from:from:date:date:received:received; s=2010; t=1399378506; bh=m/3iHPhQ9ZGwhbkw/5jfzzLttUg1fiya0WNQvsM/o6c=; b=XgHT8LYB9V4U 7rcNzau88jR9H8vEaejjgat1Xpqb64ZJUVxkt/oDHbLauRQ7WZixaI7lwn6lSt09 WtU2uci+ABDFT1DJa/QvSe22opkheBbDK5uQGLIHXdHL//ThfhZAPHz0SRQbNXFs F82ACl6sfiIFJfaCBybJzHZERCNzei8=
In-reply-to: <53687954.70903@xxxxxxxxxxxxxxxx>
References: <20140429141502.GD15985@xxxxxxxxxxxxxxx> <y0mzjj4j5pp.fsf@xxxxxxxx> <53601960.5000503@xxxxxxxxxxxxxxxx> <20140501200341.GA23705@xxxxxxxxxxxxxxx> <005101cf6834$43bbc140$cb3343c0$@internode.on.net> <20140505184224.GB26622@xxxxxxxxxxxxxxx> <53687954.70903@xxxxxxxxxxxxxxxx>
User-agent: Mutt/1.5.21 (2012-12-30)
Hi Ken,

On Tue, May 06, 2014 at 03:55:32PM +1000, Ken McDonell wrote:
> On 06/05/14 04:42, Michele Baldessari wrote:
> >...
> >>And if Frank's general suggestion below /var/log/pcp does not already catch
> >>these, I'd suggest adding /var/log/pcp/pmie/<hostname> and
> >>/var/log/pcp/pmproxy.
> >
> >If the files in there end up with *config or *log, we are covered.
> >
> > ...
> >>Also /var/log/pcp/pmwebd and /var/log/pcp/pmmgr may contain relevant info,
> >>but I'll defer to Frank who's the expert on these ones.
> >
> >Same here, if the files end with .log or .config.
> 
> Hmm, not quite
> 
> Can we add .log.prev to the suffixes list?  Sometimes the bad thing you're
> looking for was _before_ the last reboot or pmcd restart, and the .prev
> files may be helpful here.

Yep, those are already covered as we have *.log* under /var/log/pcp/
(*/*.log and */*/*.log)
 
> The pmmgr subdirectory also contains config.* files (not *.config), and a
> bunch of PCP archives in pmmgr/<hostname> similar to pmlogger/<hostname>

I had added config* but wrote *.config in my mail. The former is the one
in the code. Is that fine as it is or do I need to tweak this?

> Now the archives are a bit tricky ... it looks (I've never checked this
> before) like pmmgr _and_ pmlogger_control/pmlogger_daily may be both running
> collecting archives with the very similar, but not identical pmlogger
> configuration files.  This is the case on my workstation, but that could be
> decades away fro a default install.
> 
> I would expect that depending on local enable/disable actions by a sysadmin,
> you may have neither, one or the other, or both sets of archives.
> 
> When both are available, it is further complicated by the fact that the
> actual configuration file for the two pmloggers may have been modified by
> local sysadmin action.
> 
> I don't have a good heuristic to choose one if you want to avoid capturing
> both, but ..

I think it might be best to start off by collecting the archives of
pmlogger of the host itself. Once we observe how many sosreport we get
where having pmmgr/hostname would have been needed, we can reassess
this. My gut feeling is that, at least initially, the most used use-case
will be a moderately simple log collection for the single host. In any
case we can amend here as necessary once we have the first pcp-enabled
sosreports flowing in.

> >One of my goals with this sosreport exercise, is to create a script like
> >sarstats[1] that I can simply point at an uncompressed sosreport and
> >quickly get a decent overview of the main PCP stats collected on the
> >machine.
> 
> OK, one possible heuristic is
> 
> 1. start with metrics you need for a sarstat-like tool ... as a starter,
> check the calls to put() and putinst() in sar2pcp from the pcp rpms
> 2. process the most recent archive from each of
> /var/log/pcp/{pmlogger,pmmgr}/<hostname> and use pminfo -a <archive>.meta to
> get the metrics in that archive, cull those not in the list from 1.
> 3. choose the archive with the longest matching list of metrics from 2.
> 
> This sort of assumes the last archive in each directory is easily found
> (lexical sorting will work, pick the .meta files as these must be there and
> are unique) and that this archive is currently active (pminfo -f
> pmcd.pmlogger.archive will help ... in fact that avoids any lexical sorting
> completely and only lists the active ones).
> 
> And it assumes the logging intervals for the metrics are similar (true by
> default I think).

Thanks, I'll definitely take a look at that.

regards,
Michele
-- 
Michele Baldessari            <michele@xxxxxxxxxx>
C2A5 9DA3 9961 4FFB E01B  D0BC DDD4 DCCB 7515 5C6D

<Prev in Thread] Current Thread [Next in Thread>