pcp
[Top] [All Lists]

Re: [pcp] proc.runq.swapped

To: myllynen@xxxxxxxxxx
Subject: Re: [pcp] proc.runq.swapped
From: Nathan Scott <nathans@xxxxxxxxxx>
Date: Fri, 21 Aug 2015 02:51:22 -0400 (EDT)
Cc: pcp developers <pcp@xxxxxxxxxxx>
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <55D57D3D.5050900@xxxxxxxxxx>
References: <55D57D3D.5050900@xxxxxxxxxx>
Reply-to: Nathan Scott <nathans@xxxxxxxxxx>
Thread-index: Wc2YK1SGg1VifroNmP/y7RyQaYpsIw==
Thread-topic: proc.runq.swapped
Hi Marko,

----- Original Message -----
> Hi,
> 
> I'm using this script to see which processes have (parts of) their memory
> swapped:
> 
> #!/bin/sh
> #
> # Total swap usage as shown by free(1) and /proc/swaps does not exactly match
> # with the swap usage summed up for individual processes as done below due to
> # pages in swap cache being included in free(1) and /proc/swaps usage amount.
> 
> for p in /proc/[1-9]* ; do
>   awk '/^[0-9]+ \(/ {printf "%-8s%-18s", $1, $2; nextfile} \
>     /Swap:/ {swap+=$2} END {printf "%8.4f MB\n", swap/1024}' $p/stat $p/smaps
>     \
>       2>/dev/null ;
> done|sort -gk3|awk '{s=s+$3;print $0}END{printf "Total:%28.4f %s\n", s,
> "MB"}'

It looks like this script has a different definition of swapped to the value
that proc.runq.swapped presents.  The script is looking at smaps, and looking
for any mappings within a process listed as swapped out.  Reading through the
pmdaproc code, proc.runq.swapped counts processes where the address space is
entirely swapped out (not just parts thereof), and this is infered using only
values from /proc/pid/stat.

As to the note about free(1) above, its reporting the swapped value directly
from /proc/meminfo (swap.used metric in pcp, or mem.util.swapFree subtracted
from mem.util.swapTotal).

> On a busy test system it reports 63 (all being user-land processes) but
> proc.runq.swapped is zero. The help text says:
> 
> @ proc.runq.swapped number of processes that are swapped
> Instantaneous number of processes (excluding kernel threads) that are
> swapped, state 'SW' in ps

(looks like the text needs an update - would be more correct to say it counts
processes with zero resident memory as swapped, excluding kernel threads).

> So looks like proc.runq.swapped is currently meaningless?

Just a different meaning to the value being presented by the script, I
think, not meaningless.

> Are there any other
> ways to find out this kind of information? Is fixing proc.runq.swapped
> possible?

We have no code looking inside the smaps files at all currently in PCP -
new metrics could certainly be added to export values from there though,
its a SMOP.

I did notice going through the atop code recently that it also looks at
smaps, but only if a special option is used (-R) and it's using smaps to
extract this (and nothing else - no swap metrics from there FWLIW) ...

       PSIZE    The proportional memory size of this process (or user).
                Every process shares resident memory  with  other  proâ
                cesses.  E.g. when a particular program is started sevâ
                eral times, the code pages (text) are only loaded  once
                in memory and shared by all incarnations. Also the code
                of shared libraries is shared by  all  processes  using
                that  shared library, as well as shared memory and memâ
                ory-mapped files.   For  the  PSIZE  calculation  of  a
                process,  the  resident  memory  of  a  process that is
                shared with other processes is divided by the number of
                sharers.   This  means, that every process is accounted
                for a proportional part of  that  memory.  Accumulating
                the PSIZE values of all processes in the system gives a
                reliable impression of the total resident  memory  conâ
                sumed by all processes.
                Since gathering of all values that are needed to calcuâ
                late the PSIZE is a relatively time-consuming task, the
                'R'  key  (or  '-R'  flag)  should be active.

(we don't support this in pcp-atop yet, but we could if a PMDA was made
to export this metric - it would need similar code to that which you're
after to mimic your script above, looking at Pss: instead of Swap: tho).

cheers.

--
Nathan

<Prev in Thread] Current Thread [Next in Thread>