| To: | Ken McDonell <kenj@xxxxxxxxxxxxxxxx> |
|---|---|
| Subject: | Re: [pcp] per cpu utilisation from archive - pmval |
| From: | Allan McAleavy <allan.mcaleavy@xxxxxxxxx> |
| Date: | Thu, 19 May 2016 13:43:40 +0100 |
| Cc: | pcp developers <pcp@xxxxxxxxxxx> |
| Delivered-to: | pcp@xxxxxxxxxxx |
| Dkim-signature: | v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=o5o6y2MBKzhhk0wzmvzj8sE2GIEqy4Q6Eo47Nejoi44=; b=LqWN4CeikdtZ9Gfec12UguexVACwI2pE4fWIrtoZk2W4JJGQW2/2oun+fPbYHwsN7U 3pW8OZcHfG98SPj3dBJzTQqufSgkjVXi2nX7GAIwDKU8NhF4IKjEzq0fxO780nnYgVZ/ Tk9Ds7LrBymyv3HltaxrpFRKISZJXo8zHLna9VqEJWTJCMofqMGW/mFFUdNIc8EROB/m FCeOcuMl9rlE0NAC/IAl5IthBduolPI1UNisDjEPaQo4N6pvDZYGALFfpyVtwJ7eCeYE xRnJ3StPN1r0hIrWxBLM7hUapmzgZ7v59GerIVQ10lIokK2XxaCmI+iiCZDUhr5CkdYn afog== |
| In-reply-to: | <573DA17D.7040408@xxxxxxxxxxxxxxxx> |
| References: | <CAF6XsOcfikRQ-EJ9cM+iuHhpHEinvKaKDWm3oB27gMPwF2vcOw@xxxxxxxxxxxxxx> <573D0011.5090505@xxxxxxxxxxxxxxxx> <CAF6XsOeRrup35qqzpbB7mKH+cOdKrPHoLJENZDobn5qPWTUzfA@xxxxxxxxxxxxxx> <573DA17D.7040408@xxxxxxxxxxxxxxxx> |
|
Hi Ken As we have a default of one minute for the pmlogger , I was wanting to gather per cpu %util from the archive on the command line. So following your example where I have one second samples. Run a five second workload over 12 cores , pmlogger logging at 1 second intervals. PCP_DERIVED_CONFIG=/var/tmp/user_pct pmval -f3 kernel.pct.cpu.user -s 30 > /var/tmp/pmval.log & Âmpstat -P ALL 1 30 > /var/tmp/mpstat.log & sleep 5 ; stress -c 12 --timeout 5 cat /var/tmp/user_pct kernel.pct.cpu.user = 100 * kernel.percpu.cpu.user Take a sample from mpstat where we do some work. 12:36:03   CPU  Â%usr  %nice  Â%sys %iowait  Â%irq  %soft Â%steal Â%guest  %idle 12:36:04   all  71.38  Â0.00  Â0.44  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  28.19 12:36:04    0  96.97  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â3.03 12:36:04    1  73.00  Â0.00  Â1.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  26.00 12:36:04    2  90.00  Â0.00  Â1.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â9.00 12:36:04    3 Â100.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00 12:36:04    4  21.00  Â0.00  Â2.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  77.00 12:36:04    5 Â100.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00 12:36:04    6  93.07  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â6.93 12:36:04    7  74.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  26.00 12:36:04    8  72.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  28.00 12:36:04    9  91.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â9.00 12:36:04   Â10 Â100.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00 12:36:04   Â11 Â100.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00 12:36:04   Â12  Â1.00  Â0.00  Â3.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  96.00 12:36:04   Â13  21.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  79.00 12:36:04   Â14  Â9.90  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  90.10 12:36:04   Â15 Â100.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00 Take the 6th sample from the pmval file. metric:  Âkernel.pct.cpu.user host:   Âxxxxxxxx semantics: cumulative counter (converting to rate) units:   millisec (converting to time utilization) samples:  30 interval: Â1.00 sec Âcpu0 Âcpu1 Âcpu2 Âcpu3 Âcpu4 Âcpu5 Âcpu6 Âcpu7 Âcpu8 Âcpu9 cpu10 cpu11 cpu12 cpu13 cpu14 cpu15 Â0.000 0.000 1.000 0.000 0.000 0.000 0.000 0.000 1.000 1.000 0.000 0.000 0.000 0.000 0.000 0.000 Â0.999 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.999 0.000 0.000 0.000 0.000 0.000 0.000 Â0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 Â1.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 1.000 0.000 0.000 0.000 0.000 0.000 0.000 Â1.999 0.000 5.997 7.996 0.000 5.997 0.000 0.000 0.000 0.000 5.997 6.996 0.000 0.000 0.000 5.997 Â99.936 72.953 83.946 99.936 27.982 Â100.935 97.937 80.948 71.954 96.938 Â100.935 99.936 3.997 27.982 15.990 99.936 Â-- sample to match. Â99.933 39.973 47.968 99.933 3.997 59.960 47.968 97.934 0.999 99.933 99.933 99.933 99.933 99.933 99.933 96.935 Â99.935 63.958 48.968 99.935 40.973 82.946 9.993 50.967 0.999 99.935 99.935 99.935 99.935 99.935 99.935 99.935 Â99.935 98.935 47.969 99.935 52.965 74.951 0.000 25.983 0.000 Â100.934 99.935 99.935 99.935 99.935 99.935 99.935 Â91.951 59.968 0.999 91.951 39.979 71.962 22.988 50.973 27.985 90.952 91.951 91.951 91.951 91.951 91.951 91.951 Â0.999 0.999 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.999 0.000 0.000 0.000 0.000 0.000 0.000 ÂWe can see that these roughly match on a 1 second interval from the pmval and mpstat data.  cpu0 cpu1 cpu2 cpu3 cpu4 cpu5 cpu6 cpu7 cpu8 cpu9 cpu10 cpu11 cpu12 cpu13 cpu14 cpu15 96.97 73 90 100 21 100 93.07 74 72 91 100 100 1 21 9.9 100   (MPSTAT DATA ) 99.936 72.953 83.946 99.936 27.982 100.935 97.937 80.948 71.954 96.938 100.935 99.936 3.997 27.982 15.99 99.936 Â(PMVAL DATA - realtime) Taking the data from the archive. PCP_DERIVED_CONFIG=/var/tmp/user_pct pmval -w8 -f2 kernel.pct.cpu.user -a /var/log/pcp/pmlogger/udapp40a/20160519.10.50.0        Âcpu0   cpu1   cpu2   cpu3   cpu4   cpu5   cpu6   cpu7   cpu8   cpu9  Âcpu10  Âcpu11  Âcpu12  Âcpu13  Âcpu14  Âcpu15 12:36:04.531  81.30  Â69.90  Â85.20  Â86.10  Â10.50  Â86.10  Â79.40  Â61.20  Â68.90  Â76.60  Â86.10  Â86.10   1.00  Â10.50   0.00  Â85.20 12:36:05.531  98.80  Â36.70  Â49.90  Â99.00  Â16.90  Â65.40  Â51.70  Â95.90   4.10  Â98.50  Â99.00  Â99.00  Â81.60  Â95.60  Â94.10  Â97.00 12:36:06.531 Â100.00  Â51.30  Â37.40  100.00  Â39.00  Â89.90  Â24.00  Â58.70   1.00  100.00  Â99.00  100.00  Â99.50  Â99.00  100.00  Â98.90 12:36:07.531 Â100.00  Â96.30  Â60.10  100.00  Â40.10  Â63.20   0.90  Â38.70   0.00  100.00  100.00  100.00  100.00  100.00  100.00  100.00 12:36:08.531 Â100.00  Â69.20   3.40  100.00  Â57.30  Â88.90  Â22.10  Â40.90  Â17.30  100.00  100.00  100.00  100.00  100.00  100.00  100.00 12:36:09.531  13.60  Â13.30   0.00  Â13.60   2.20   3.50   0.90  Â11.30  Â10.40  Â14.60  Â14.60  Â13.60  Â13.60  Â14.60  Â13.60  Â13.60 And looking at one CPU CPU 7 12:36:04.531 61.20 12:36:05.531 95.90 12:36:06.531 58.70 12:36:07.531 38.70 12:36:08.531 40.90 12:36:09.531 11.30 grep "   Â7" mpstat.log.test1 12:36:04    7  74.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  26.00 12:36:05    7  98.00  Â0.00  Â2.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00 12:36:06    7  52.48  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  47.52 12:36:07    7  31.31  Â0.00  Â0.00  Â1.01  Â0.00  Â0.00  Â0.00  Â0.00  67.68 12:36:08    7  51.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  49.00 So I am happy with the output as that shows similar from pmlog given sampling. Test2 - Default setting 1 min. mpstat data 13:26:55   CPU  Â%usr  %nice  Â%sys %iowait  Â%irq  %soft Â%steal Â%guest  %idle 13:26:56   all  67.56  Â0.00  Â0.31  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  32.12 13:26:56    0  98.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â2.00 13:26:56    1  88.89  Â0.00  Â2.02  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â9.09 13:26:56    2  97.03  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â2.97 13:26:56    3  99.01  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.99 13:26:56    4  97.98  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â2.02 13:26:56    5  96.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â4.00 13:26:56    6  98.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â2.00 13:26:56    7  74.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  26.00 13:26:56    8  46.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  54.00 13:26:56    9  92.93  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â7.07 13:26:56   Â10  Â0.00  Â0.00  Â2.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  98.00 13:26:56   Â11 Â100.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00 13:26:56   Â12  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00 Â100.00 13:26:56   Â13  94.95  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â5.05 13:26:56   Â14  Â0.00  Â0.00  Â1.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  99.00 13:26:56   Â15  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00  Â0.00 Â100.00 Âcpu0 Âcpu1 Âcpu2 Âcpu3 Âcpu4 Âcpu5 Âcpu6 Âcpu7 Âcpu8 Âcpu9 cpu10 cpu11 cpu12 cpu13 cpu14 cpu15 Â0.998 1.996 0.998 0.998 0.000 0.000 0.000 0.000 0.000 0.000 1.996 2.994 0.000 0.998 0.000 0.000 Â0.999 0.999 0.000 0.999 0.000 0.000 0.000 0.000 0.999 0.999 2.998 1.999 0.000 0.000 0.000 0.000 Â0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.999 0.000 0.000 0.000 0.000 0.000 0.000 Â0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 Â0.000 0.000 0.000 1.999 0.000 0.000 0.000 0.000 0.000 0.999 0.000 2.998 0.000 0.000 0.000 0.000 Â100.319 90.387 Â100.319 Â100.319 99.326 98.333 Â100.319 76.481 48.670 94.360 0.000 99.326 0.000 96.346 0.000 0.000 -- our data ( 1st sample ) Â99.923 97.925 99.923 99.923 99.923 99.923 99.923 99.923 99.923 99.923 0.000 99.923 3.997 99.923 2.998 0.999 Â81.944 97.933 99.932 63.956 99.932 99.932 99.932 99.932 99.932 99.932 0.999 99.932 17.988 99.932 0.999 34.976 Â5.996 98.931 54.962 40.971 48.966 99.930 10.992 99.930 99.930 59.958 88.938 Â100.930 99.930 99.930 89.937 99.930 Â1.999 96.938 41.973 74.952 52.966 96.938 0.000 95.939 95.939 20.987 96.938 95.939 95.939 96.938 96.938 96.938 Â0.999 0.999 0.000 1.999 0.000 0.000 0.000 0.000 0.999 0.000 0.999 0.999 0.999 0.000 0.000 0.000 Â98  88.89 97.03 Â99.01 Â97.98 96  98  74 Â46 92.93 0  ÂÂ100 Â0 94.95 0 0  Â(MPSTAT DATA ) Â100.319 90.387 Â100.319 Â100.319 99.326 98.333  100.319 76.481 48.670 94.360 Â0.000 99.326 0.000 96.346 0.000 0.000 Â(PMVAL DATA - realtime) So we match ok as sampling at 1 second.  PCP_DERIVED_CONFIG=/var/tmp/user_pct pmval -w8 -f2 kernel.pct.cpu.user -a 20160519.13.26.0 From the archive. ( started test at 13:26:51 - 13:27:20)             ÂÂÂcpu0   cpu1   cpu2   cpu3   cpu4   cpu5   cpu6   cpu7   cpu8   cpu9   cpu10  cpu11   cpu12  cpu13   cpu14  cpu15 13:26:51.759  Â5.10   8.40   6.90   6.40   6.70   8.30   5.20   8.00   7.50   6.50   3.50   8.80   3.70   8.30   3.20   4.00 13:26:52.759  Â5.10   8.40   6.90   6.50   6.70   8.30   5.20   7.90   7.40   6.60   3.50   8.80   3.70   8.30   3.20   3.90 13:26:53.759  Â5.10   8.40   6.90   6.50   6.80   8.30   5.20   7.90   7.50   6.50   3.40   8.80   3.60   8.30   3.20   4.00 13:26:54.759  Â5.10   8.40   6.80   6.50   6.70   8.30   5.20   7.90   7.40   6.60   3.50   8.80   3.70   8.30   3.20   3.90 13:26:55.759  Â5.10   8.40   6.90   6.50   6.70   8.40   5.20   7.90   7.50   6.50   3.50   8.80   3.70   8.30   3.20   3.90 13:26:56.759  Â5.00   8.40   6.90   6.50   6.70   8.30   5.20   8.00   7.40   6.60   3.40   8.70   3.60   8.30   3.10   4.00 13:26:57.759  Â5.10   8.50   6.80   6.50   6.70   8.30   5.20   7.90   7.50   6.60   3.50   8.80   3.70   8.30   3.20   3.90 13:26:58.759  Â5.10   8.40   6.90   6.40   6.80   8.30   5.20   7.90   7.50   6.50   3.50   8.80   3.70   8.20   3.20   3.90 13:26:59.759  Â5.10   8.40   6.90   6.50   6.70   8.30   5.20   7.90   7.40   6.60   3.50   8.80   3.60   8.30   3.20   4.00 13:27:00.759  Â5.10   8.40   6.80   6.50   6.70   8.30   5.20   7.90   7.50   6.50   3.40   8.80   3.70   8.30   3.20   3.90 13:27:01.759  Â5.10   8.40   6.90   6.50   6.70   8.30   5.20   8.00   7.40   6.60   3.50   8.80   3.70   8.30   3.20   3.90 13:27:02.759  Â5.10   8.40   6.90   6.50   6.70   8.30   5.20   7.90   7.50   6.50   3.50   8.80   3.60   8.30   3.20   4.00 13:27:03.759  Â5.00   8.40   6.90   6.50   6.80   8.30   5.20   7.90   7.40   6.60   3.40   8.80   3.70   8.30   3.10   3.90 13:27:04.759  Â5.10   8.40   6.80   6.50   6.70   8.30   5.20   7.90   7.50   6.50   3.50   8.70   3.70   8.30   3.20   3.90 13:27:05.759  Â5.10   8.40   6.90   6.40   6.70   8.30   5.20   7.90   7.40   6.60   3.50   8.80   3.70   8.30   3.20   4.00 13:27:06.759  Â5.10   8.40   6.90   6.50   6.70   8.30   5.20   8.00   7.50   6.50   3.40   8.80   3.60   8.20   3.20   3.90 13:27:07.759  Â5.10   8.40   6.80   6.50   6.70   8.30   5.20   7.90   7.40   6.60   3.50   8.80   3.70   8.30   3.20   3.90 13:27:08.759  Â5.10   8.40   6.90   6.50   6.80   8.30   5.20   7.90   7.50   6.50   3.50   8.80   3.70   8.30   3.20   4.00 13:27:09.759  Â5.10   8.40   6.90   6.50   6.70   8.30   5.30   7.90   7.40   6.60   3.40   8.80   3.60   8.30   3.10   3.90 13:27:10.759  Â5.00   8.40   6.80   6.50   6.70   8.30   5.20   7.90   7.50   6.50   3.50   8.80   3.70   8.30   3.20   3.90 13:27:11.759  Â5.10   8.40   6.90   6.50   6.70   8.30   5.20   8.00   7.40   6.60   3.50   8.80   3.70   8.30   3.20   4.00 13:27:12.759  Â5.10   8.40   6.90   6.40   6.70   8.30   5.20   7.90   7.50   6.50   3.40   8.70   3.60   8.30   3.20   3.90 13:27:13.759  Â5.10   8.40   6.80   6.50   6.80   8.30   5.20   7.90   7.50   6.60   3.50   8.80   3.70   8.20   3.20   4.00 13:27:14.759  Â5.10   8.40   6.90   6.50   6.70   8.30   5.20   7.90   7.40   6.50   3.50   8.80   3.70   8.30   3.20   3.90 13:27:15.759  Â5.10   8.40   6.90   6.50   6.70   8.30   5.20   7.90   7.50   6.60   3.40   8.80   3.60   8.30   3.20   3.90 13:27:16.759  Â5.10   8.40   6.90   6.50   6.70   8.30   5.20   8.00   7.40   6.60   3.50   8.80   3.70   8.30   3.10   4.00 13:27:17.759  Â5.00   8.40   6.80   6.50   6.70   8.30   5.20   7.90   7.50   6.50   3.50   8.80   3.70   8.30   3.20   3.90 13:27:18.759  Â5.10   8.40   6.90   6.50   6.80   8.30   5.20   7.90   7.40   6.60   3.40   8.80   3.60   8.30   3.20   3.90 13:27:19.759  Â5.10   8.40   6.90   6.50   6.70   8.30   5.20   7.90   7.50   6.50   3.50   8.70   3.70   8.30   3.20   4.00 13:27:20.759  Â5.10   8.40   6.80   6.40   6.70   8.30   5.20   7.90   7.40   6.60   3.50   8.80   3.70   8.30   3.20   3.90 So from above is where I was having the issue converting this to a per CPU utilisation. Thanks Al On Thu, May 19, 2016 at 12:20 PM, Ken McDonell <kenj@xxxxxxxxxxxxxxxx> wrote: On 19/05/16 21:04, Allan McAleavy wrote: |
| <Prev in Thread] | Current Thread | [Next in Thread> |
|---|---|---|
| ||
| Previous by Date: | Re: [pcp] Oracle connection debugging (was Re: Handling Oracle PMDA Latencies), Ken McDonell |
|---|---|
| Next by Date: | Re: [pcp] Oracle connection debugging (was Re: Handling Oracle PMDA Latencies), Marko Myllynen |
| Previous by Thread: | Re: [pcp] per cpu utilisation from archive - pmval, Ken McDonell |
| Next by Thread: | [Bug 1336808] New: RFE: Provide uncached per-process memory metrics, bugzilla |
| Indexes: | [Date] [Thread] [Top] [All Lists] |