>From 352ae5a5110aa7409f4eab78455b803ca8b264eb Mon Sep 17 00:00:00 2001 From: Marko Myllynen Date: Wed, 29 Apr 2015 16:40:52 +0300 Subject: [PATCH] Describe Hotproc in the Quick Guide --- man/html/guide.html | 50 +++++++++++++++++++++++++++++++++----------------- 1 files changed, 33 insertions(+), 17 deletions(-) diff --git a/man/html/guide.html b/man/html/guide.html index a143655..08a26a2 100644 --- a/man/html/guide.html +++ b/man/html/guide.html @@ -33,6 +33,7 @@

Performance Metrics Inference @@ -103,7 +104,7 @@ -

To enable centralized archive log collection on the monitoring host, its pmlogger is configured to fetch performance metrics from collector hosts. Add each collector host to the pmlogger configuration file /etc/pcp/pmlogger/control and then restart the pmlogger service on the monitoring host. +

To enable centralized archive log collection on the monitoring host, its pmlogger is configured to fetch performance metrics from collector hosts. Add each collector host to the pmlogger configuration file /etc/pcp/pmlogger/control and then restart the pmlogger service on the monitoring host.

Enable recording of metrics from remote host acme.com:
@@ -113,7 +114,7 @@

-

Checks for remote log collection will be done every half an hour. You may also wish to run /usr/libexec/pcp/bin/pmlogger_check -V -C (on Fedora/RHEL) or /usr/lib/pcp/bin/pmlogger_check -V -C (on Debian/Ubuntu) manually (service restart above issues this command internally). +

Checks for remote log collection will be done every half an hour. You may also wish to run /usr/libexec/pcp/bin/pmlogger_check -V -C (on Fedora/RHEL) or /usr/lib/pcp/bin/pmlogger_check -V -C (on Debian/Ubuntu) manually (service restart above issues this command internally).

Note that a default configuration file (config.acme.com above) will be generated if it does not exist already. This process is optional (a custom configuration for each host can be provided instead), see the pmlogconf(1) manual page for details on this. @@ -200,7 +201,7 @@ Display detailed information about a performance metric and its current values:<

-Monitor live disk write operations per partition with two second interval using fixed point notation (use -i instance to list only certain metrics and -r for raw values):
+Monitor live disk write operations per partition with two second interval using fixed point notation (use -i instance to list only certain metrics and -r for raw values):

$ pmval -t 2sec -f 3 disk.partitions.write -h acme.com

@@ -235,7 +236,7 @@ Monitor system metrics in an iostat like fashion with two second interval:

-Monitor performance metrics with a GUI application with two second default interval from two different hosts. Use File->New Chart to select metrics to be included in a new view and use File->Open View to use a predefined view:
+Monitor performance metrics with a GUI application with two second default interval from two different hosts. Use File->New Chart to select metrics to be included in a new view and use File->Open View to use a predefined view:

$ pmchart -t 2sec -h acme1.com -h acme2.com @@ -243,7 +244,7 @@ Monitor performance metrics with a GUI application with two second default inter

Retrospective Performance Analysis

-
PCP archive logs are located under /var/log/pcp/pmlogger/hostname, and the archive names indicate the date they cover. Archives are self-contained, and machine-independent so can be transfered to any machine for offline analysis. +
PCP archive logs are located under /var/log/pcp/pmlogger/hostname, and the archive names indicate the date they cover. Archives are self-contained, and machine-independent so can be transfered to any machine for offline analysis.

    @@ -275,14 +276,14 @@ Display detailed information about a performance metric at the time when an arch

    -Dump past disk write operations per partition in an archive using fixed point notation (use -i instance to list only certain metrics and -r for raw values):
+Dump past disk write operations per partition in an archive using fixed point notation (use -i instance to list only certain metrics and -r for raw values):

$ pmval -f 3 disk.partitions.write -a acme.com/20140902

    -Replay past disk write operations per partition in an archive with two second interval using fixed point notation between 9 AM and 10 AM (use full dates with syntax like @"2014-08-20 14:00:00"):
+Replay past disk write operations per partition in an archive with two second interval using fixed point notation between 9 AM and 10 AM (use full dates with syntax like @"2014-08-20 14:00:00"):

$ pmval -d -t 2sec -f 3 disk.partitions.write -S @09:00 -T @10:00 -a acme.com/20140902

@@ -303,7 +304,7 @@ Dump past CPU load, memory usage, and disk write operations per partition in an

    -Summarize differences in past performance metrics between two archives, comparing 2 AM / 3 AM in the first archive to 9 AM / 10 AM in the second archive (grep for '+' to quickly see values which were zero during the first period):
+Summarize differences in past performance metrics between two archives, comparing 2 AM / 3 AM in the first archive to 9 AM / 10 AM in the second archive (grep for '+' to quickly see values which were zero during the first period):

$ pmdiff -S @02:00 -T @03:00 -B @09:00 -E @10:00 acme.com/20140902 acme.com/20140901

@@ -338,7 +339,7 @@ Dump past system metrics in a

    -Replay performance metrics with a GUI application with two second default interval in an archive between 9 AM and 10 AM. Use File->New Chart to select metrics to be included in a new view and use File->Open View to use a predefined view:
+Replay performance metrics with a GUI application with two second default interval in an archive between 9 AM and 10 AM. Use File->New Chart to select metrics to be included in a new view and use File->Open View to use a predefined view:

$ pmchart -t 2sec -S @09:00 -T @10:00 -a acme.com/20140902

@@ -353,7 +354,7 @@ Merge several archives as a new combined archive (see the manual page how to wri
Visualizing iostat and sar Data
-
iostat and sar data can be imported as PCP archives which then allows inspecting and visualizing the data with PCP tools. The iostat2pcp(1) importer is in the pcp-import-iostat2pcp package and the sar2pcp(1) importer is in the pcp-import-sar2pcp package. +
iostat and sar data can be imported as PCP archives which then allows inspecting and visualizing the data with PCP tools. The iostat2pcp(1) importer is in the pcp-import-iostat2pcp package and the sar2pcp(1) importer is in the pcp-import-sar2pcp package.

    @@ -366,7 +367,7 @@ Import iostat data to a new PCP archive and visualize it:
@@ -375,7 +376,7 @@ Import sar data from an existing sar archive to a new PCP archive and visualize
Process Level Performance Monitoring
-
PCP provides details of each running process via the standard PCP interfaces and tools on the localhost but due to security and performance considerations, most of the process related information is not stored in archive logs by default. +
PCP provides details of each running process via the standard PCP interfaces and tools on the localhost but due to security and performance considerations, most of the process related information is not stored in archive logs by default. Also for security reasons, only root can access some details of running processes of other users.
Custom application instrumentation is possible with the Memory Mapped Value (MMV) PMDA. @@ -397,7 +398,7 @@ Monitor the number of open file descriptors of the process 1234:

    -Import sar data from an existing sar archive to a new PCP archive and visualize it (sar logs are under /var/log/sysstat on Debian/Ubuntu):
+Import sar data from an existing sar archive to a new PCP archive and visualize it (sar logs are under /var/log/sysstat on Debian/Ubuntu):

$ sar2pcp /var/log/sa/sa15 sar.pcp
$ pmchart -t 2sec -a sar.pcp

    -Monitor the CPU time, memory usage (RSS), and the number of threads of the process 1234 (-h local: is a workaround needed for the time being):
+Monitor the CPU time, memory usage (RSS), and the number of threads of the process 1234 (-h local: is a workaround needed for the time being):

$ pmdumptext -h local: -t 2sec 'proc.psinfo.utime[1234]' 'proc.memory.rss[1234]' 'proc.psinfo.threads[1234]'

@@ -416,6 +417,21 @@ Display the number of running processes on 2014-08-20 14:00:

+ +
Monitoring “Hot” Processes with Hotproc
+ +
It is also possible to monitor “hot” or “interesting” processes by name with PCP 3.10.5 or later, for example all processes of which command name is java or python. This monitoring of “hot” processes can also be enabled or disabled based on certain criterias or from the command line on the fly. The metrics will be available under the namespace hotproc. +
Configuring processes to be monitored contantly using the hotproc namespace can be done using the configuration file /var/lib/pcp/pmdas/proc/hotproc.conf - see the pmdaproc(1) manual page for details. This allows monitoring these processes regardless of their PIDs and also logging the metrics easily. + + + +

    +Enable monitoring of all Java instances on the fly and display all the collected metrics:
+
# pmstore hotproc.control.config 'fname == "java"' +
# pminfo -f hotproc +
+ +
Application Instrumentation
@@ -443,7 +459,7 @@ Display the number of running processes on 2014-08-20 14:00:
-
To enable the monitoring host to run PMIE for collector hosts, add each host to the /etc/pcp/pmie/control configuration file. +
To enable the monitoring host to run PMIE for collector hosts, add each host to the /etc/pcp/pmie/control configuration file.

   Enable monitoring of metrics from remote host acme.com: @@ -506,12 +522,12 @@ This example shows a PMIE script, checks its syntax, runs it against an archive,
PCP PMDAs offer a way for administrators and developers to customize and extend the default PCP installation. The pcp-libs-devel package contains all the needed development related examples, headers, and libraries. New PMDAs can easily be added, below is a quick list of references for starting development:
-
Some examples exist below /var/lib/pcp/pmdas/ - the simple, sample, and txmon PMDAs are easy to read PMDAs. +
Some examples exist below /var/lib/pcp/pmdas/ - the simple, sample, and txmon PMDAs are easy to read PMDAs.

The simple PMDA provides implementations in C, Perl and Python.

-
A simple command line monitor tool is /usr/share/pcp/demos/pmclient (C language).
-
Good initial Python monitor examples are /usr/libexec/pcp/bin/pcp/pcp-* (Fedora/RHEL) or /usr/lib/pcp/bin/pcp-* (Debian/Ubuntu). +
A simple command line monitor tool is /usr/share/pcp/demos/pmclient (C language).
+
Good initial Python monitor examples are /usr/libexec/pcp/bin/pcp/pcp-* (Fedora/RHEL) or /usr/lib/pcp/bin/pcp-* (Debian/Ubuntu).

Slightly more complex examples are the pmiostat, pmatop, pmcollectl commands.

-- 1.7.1