pcp
[Top] [All Lists]

Re: [pcp] Qn on supporting metrics from multiple hosts in pmstat, pmchar

To: Jun Wang <junwang123@xxxxxxxxx>
Subject: Re: [pcp] Qn on supporting metrics from multiple hosts in pmstat, pmchart and pmie tools
From: Nathan Scott <nathans@xxxxxxxxxx>
Date: Thu, 6 Dec 2012 21:27:06 -0500 (EST)
Cc: pcp@xxxxxxxxxxx
In-reply-to: <CANHDG4YCqwJ_6BHAc=oEMGUo7GYt_E6fLUajNEi7o6xJ-FhawQ@xxxxxxxxxxxxxx>
Reply-to: Nathan Scott <nathans@xxxxxxxxxx>
Hi Jun,

----- Original Message -----
> 
> Hi PCP Folks,
> 
> Being a newbie on PCP, I have questions on monitoring metrics from
> multiple collector hosts. On the monitoring side, with tools like
> pmchart, pmie and pmstat, what's the strategy to monitor metrics
> from multiple collector nodes? To my understanding, each pmcd,
> together with the corresponding pmdas, runs and manages one
> collector node.

That's correct.

> I'm reading PCP on Linux user's guide on
> http://www.damtp.cam.ac.uk/cosmos/private/documentation/SGI/propack/PCP_LinuxUAG/pdf/PCP_LinuxUAG.pdf
> .
> 
> pmstat has an option -H which allows users to specify multiple hosts.
> I verifies that it does print out stats from multiple hosts, one
> after the other when hosts are specified with -H.

*nod*.  You can also use multiple -h options on the command line to
pmstat, which results in an extra initial column showing the node:

$ pmstat -h smash -h verge -t 1
@ Fri Dec  7 13:13:04 2012
node    loadavg               memory      swap        io    system         cpu
          1 min   swpd   buff  cache   pi   po   bi   bo   in   cs  us  sy  id
smash      1.45      0  4015m  8746m    0    0    0    0  493 1246   0   1  99
verge      0.30      0 404512  2213m    0    0    0    0  291  358   1   1  98
smash      1.45      0  4025m  8746m    0    0    0    0  575 1471   1   0  99
verge      0.30      0 404648  2213m    0    0    0    0  264  296   2   1  97

> pmie has an -h option which allow users to specify only one host to
> monitor.

Strictly speaking, that specifies the default host.  Others can
be specified directly, in the pmie language, for individual rules
or parts of rules.  For rules with no host explicitly specified,
the default is used.  If no default is specified, localhost will
be used.

> However, it appears that hosts can also be specified in the
> pmie specification language to create per-host and cross-host rules.

That's right.

> Does this implicitly trigger the communication between pmie and the
> pmcd(s) on the corresponding hosts that are specified in the pmie
> spec language?

Yes.

> On the other hand, there appears to be discussions on
> starting one pmie, on the monitor side, for each collector host.
> What's the recommended approach?

Whatever works best for you.  I've seen production deployments where
either, and both strategies have been used.

> pmchart also has an -h option allowing users to specify only one host

Like with pmie before, that's for the default host(s).  And like with
pmstat, you can give multiple -h options.

> to monitor. However, there are discussions on hosts in the pmchart
> man page and it is claimed that pmchart can display metrics from
> multiple hosts. How do I specify those hosts? Is it available on the
> menu of the GUI? What's the relation between those hosts and the
> ONLY host specified with -h option?

Using this as a guide:
http://oss.sgi.com/projects/pcp/pcp-gui.git/man/html/lab.pmchart.html
(from the PCP Manual link on the bottm left of the PCP project page)

On the third screenshot (Open View) there's a "Host:" drop down, and
a new host button between Open and Cancel.  The drop down menu will
contain all hosts seen so far (so, all -h options, and any hostnames
entered via the UI).  The new host button pops up another window that
will allow you to enter additional host names.

FWIW, when running in "archive mode" (historical data), the UI is all
the same, except instead of "Hosts:" the user can enter additional PCP
archives from which historical data is sourced.  The archives have a
one-to-one mapping with hosts (i.e. an archive records data from one
host and one host only).

If you scroll further down the above URL to the sixth screenshot, you
will see another Host button on the New Chart window - that has the
same effect as the new host button on the Open View window earlier.

> Also, if I define a new metrics domain on a collector, a new PMDA and
> new configuration line(s) in pmcd to enable it, how do I make it
> available to be selected from pmchart menu? What about multiple

As soon as the PMDA is installed (via its Install script, usually),
its metrics become immediately available in the client tools like
pmchart, with no other intervention required (client tools do auto-
discovery via the metric names and their associated metadata).

> metrics domains on multiple collector hosts? Can they be displayed
> with one pmchart application but multiple views on the monitor side?

Yes.  It's even possible in pmchart to have a single chart that has
individual plots from different hosts.

Good questions!  :)  The PCP man pages are extensive too - so on any
host that you've installed pcp and pcp-gui on, you will also be able
to browse through:

$ man pmstat
$ man pmie
$ man pmchart
$ man pmcd

Several of the command line option (e.g. time window specification, for
archives) are also documented on the PCPIntro man page, might be of use
too.  Enjoy!

cheers.

--
Nathan

<Prev in Thread] Current Thread [Next in Thread>