Hi Mark,
----- "Mark Nelson" <mark@xxxxxxxxxxx> wrote:
> Hi,
>
> We are currently running collectl on our nodes here and collect data
> from the moab/torque logs to gather statistics about what is happening
> on our clusters. I currently generate heatmaps and various other
> charts
> using some tools I've written (they are open source so let me know if
> you are interested). I was wondering if PCP might provide us with any
> additional statistics beyond what we are getting via collectl and
> moab.
I've not come across moab before, can't really comment there (google
is not helping me - got a pointer?). I have had a look at collectl in
the past though, which is a 6000+ line perl script designed primarily
(soley?) for Linux kernel metrics. It lays claim to being "better than
sar" which no doubt it has achieved.
PCP is a cross-platform extensible framework. So, it supports kernel
metrics from Linux, Windows, Solaris, Mac OS X, and other kernels.
It has an extensible architecture, is not focussed only on the kernel,
but rather the entire system - there are collection mechanisms for many
subsystems that ship out-of-the-box with PCP, and an ideal production
deployment would extend that set with custom site-specific metrics.
Some of the areas PCP ships optional metrics for are mysql, named, samba,
apache, kvm, netfilter, vmware, sendmail, web logs, lmsensors, cisco,
lustre, pdns, zimbra, memcache, postfix... so, to answer your question,
yes PCP will provide you with additional statistics beyond what you have
now with collectl.
FWIW, even out-of-the-box PCP supports many hundreds of Linux kernel
metrics (PCP has an architecture which lends itself to extension):
$ cd src/pmdas/linux
$ pminfo -n root_linux | wc -l
744
So, thats 744 kernel metrics out of the box, the set grows if you make
use of cgroups, and some kernel metrics are in separate PMDAs - infiniband,
Lustre, etc.
> I've included some samples below of the kind of information that I'm
> gathering and plotting now:
>
> http://www.msi.umn.edu/~mark/msica/mirror_20090928-20100927.png
> http://www.msi.umn.edu/~mark/msica/2GB-block_64MB_directIO_posix_nocache.png
Interesting stuff. We've spoken about heat maps before and its an area of
interest for some - the native PCP charting tool, pmchart, doesn't support
heatmaps at this time, but Qwt on which it is built does ... so its a Simple
Matter of Coding to add that functionality into pmchart - I'm interested in
seeing someone tackle that and attempt to solve it in a generic way (i.e.
heatmaps for arbitrary metrics & completely runtime configurable like the
other chart types there).
cheers.
--
Nathan
|
