pcp
[Top] [All Lists]

iostat2pcp lives

To: Mark Goodwin <goodwinos@xxxxxxxxx>
Subject: iostat2pcp lives
From: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Date: Fri, 30 Jul 2010 09:36:05 +1000
Cc: pcp@xxxxxxxxxxx
In-reply-to: <4C4E53F0.1060408@xxxxxxxxx>
References: <1031318859.1247151280192297713.JavaMail.root@xxxxxxxxxxxxxxxxxx> <1280198413.3348.6.camel@xxxxxxxxxxxxxxxx> <4C4E53F0.1060408@xxxxxxxxx>
Reply-to: kenj@xxxxxxxxxxxxxxxx
On Tue, 2010-07-27 at 13:35 +1000, Mark Goodwin wrote:
> On 07/27/2010 12:40 PM, Ken McDonell wrote:
> > On Tue, 2010-07-27 at 10:58 +1000, nathans@xxxxxxxxxx wrote:
> >> ...
> >>
> >> Looks like -t option (at least to sysstat's iostat) will print the
> >> timestamp for each sample.  And it appears the S_TIME_FORMAT env
> >> variable set to "ISO" will give timestamp with timezone info.
> >
> > OK that's encouraging.
> >
> > Mark, would you like to send me a potted collection of sample iostat
> > output that you apparently already have (I'd only need the first couple
> > of samples for each) and I'll start thinking about notsopsychic2pcp ...
> 
> ok, attached is some iostat data and also some mpstat data (both
> with obfuscated hostnames). ...

iostat2pcp is done, ... as in

sed -e 1d -e /END:/d dm-8-iostat-1007271040.log \
| iostat2pcp -v -S 10:40:20 -t 1 -Z +1200 - dm

I need the sed to strip the header and trailer lines that Mark's script
added around the iostat output.  The start time (-S), sample interval
(-t) and timezone (-Z) have to be added by a human as this information
is not in the bare iostat output (iostat -t _is_ sufficient to allow
iostat2pcp to determine the -S and -t values, and iostat -t with
$S_TIME_FORMAT=ISO in the environment allows autodetection of -Z also).

$ pminfo -a dm
disk.dev.avactive
disk.dev.write_bytes
disk.dev.read_bytes
disk.dev.write
disk.dev.read
disk.dev.write_merge
disk.dev.read_merge
kernel.all.cpu.idle
kernel.all.cpu.steal
kernel.all.cpu.wait.total
kernel.all.cpu.sys
kernel.all.cpu.nice
kernel.all.cpu.user

And there are some busy little disks in this archive ...

$ cat <<'End-of-File' | pmie -a dm -z -t 1sec
> some_inst (
>     all_sample ( disk.dev.avactive @0..59 > 0.95 )
> ) -> print 60 sec "Busy:" " %i";
> End-of-File
pmie: timezone set to local timezone of host bozo
print Tue Jul 27 10:41:21 2010: Busy: sds sdac sdbe sdbo dm-8 sdby sdbz sdca 
sdcb
print Tue Jul 27 10:42:32 2010: Busy: sds sdac sdbe sdbo dm-8 sdby sdbz sdca 
sdcb
print Tue Jul 27 10:43:45 2010: Busy: sds sdac sdbe sdbo dm-8 sdby sdbz sdca 
sdcb
print Tue Jul 27 10:45:04 2010: Busy: sds sdac sdbe sdbo dm-8 sdby sdbz sdca 
sdcb
print Tue Jul 27 10:46:24 2010: Busy: sds sdac sdbe sdbo dm-8 sdby sdbz sdca 
sdcb
print Tue Jul 27 10:47:24 2010: Busy: sds sdac sdbe sdbo dm-8 sdby sdbz sdca 
sdcb
print Tue Jul 27 10:48:24 2010: Busy: sds sdac sdbe sdbo dm-8 sdby sdbz sdca 
sdcb
print Tue Jul 27 10:49:37 2010: Busy: sds sdac sdbe sdbo dm-8 sdby sdbz sdca 
sdcb
print Tue Jul 27 10:51:06 2010: Busy: sds sdac sdbe sdbo dm-8 sdby sdbz sdca 
sdcb
print Tue Jul 27 10:53:02 2010: Busy: sds sdac sdbe sdbo dm-8 sdby sdbz sdca 
sdcb
print Tue Jul 27 10:54:24 2010: Busy: sds sdac sdbe sdbo dm-8 sdby sdbz sdca 
sdcb
print Tue Jul 27 10:55:40 2010: Busy: sds sdac sdbe sdbo dm-8 sdby sdbz sdca 
sdcb
print Tue Jul 27 10:56:55 2010: Busy: sds sdac sdbe sdbo dm-8 sdby sdbz sdca 
sdcb
[Thu Jul 29 23:23:19] pmie(8548) Info: evaluator exiting

I would have been done 2 days ago except for timezone madness ... Perl's
Date:: module and libc and PCP _cannot_ agree on timezone format,
forcing me to do some gross hackery ... the -Z +1200 from the command
line is used in all the Perl code, but I need the (really bogus)
PCP-12:00:00 timezone to get the timestamps in the PCP archives to be
interpreted correctly.  None of the other million or so variations I
tried worked in all test cases.

$ pmdumplog -z -L dm
Note: timezone set to local timezone of host "somehost.somewhere.com" from 
archive

Log Label (Log Format Version 2)
Performance metrics from host somehost.somewhere.com
  commencing Tue Jul 27 10:40:21.000 2010
  ending     Tue Jul 27 10:57:11.000 2010
Archive timezone: PCP-12:00:00
PID for pmlogger: 8754

I have a bit more documentation to clean up (thanks Greg!) and I'll do
another bulk commit later today after I get through my domestic chores.

Cheers, Ken.

ps I'm well over this for a while, so mpstat2pcp will have to wait on
the back burner until I have the strength to jump back into Perl.

pps For most of what I do, I still prefer the world of Aho, Weinberger
and Kernighan although Perl has been an interesting diversion.



<Prev in Thread] Current Thread [Next in Thread>