On Wed, 25 Sep 2002, Mike Mason wrote:
> Mark,
> ...
>
> >
> >
> >>(new) Per process map info from /proc/<pid>/maps
> >> proc.memory.maps
> >
> >
> > Seems OK. Did you need this to be specifically the same as cat /proc/*/maps
> > i.e. a very long string? Does it still work OK if the maps for a particular
> > process exceeds the copyout limit (one page)?
> >
> >
>
> This is needed by libgtop. I couldn't think of any other way to
> represent the maps in PCP. As far as I know, there's no way to
> represent instances within instances. Have you ever considered adding
> the capability?
>
> I don't think the copyout limit is an issue for PCP. There's code in
> proc/array.c to handle maps files greater than 4K. We just read it, so
> I don't think we need to do anything special.
Seems like there are several issues sort of intermingled here ...
1. The copyout problem ... I think there remains a systemic Linux issue
with any /proc "file" that is longer than your page size du jour ...
if the underlying data is subject to change, there is no way a user
space app (cat(1) is the simplest example) is guaranteed to see a
consistent view ... you may miss whole lines, you may see some lines
more than once.
If this assertion is not correct, I'd like to understand why.
This is pretty much independent of what PCP chooses to do with the
data.
2. The n-dimensional instance domain issue ... this is a PCP matter.
The set-based {metric} x {instance} data model is fundamental to
PCP and something I would defend to the death. On the other hand
from the outset, we strove for simplicity, light-weight collection
cost and fought off all suggestions for premature optimizations.
This is how we got to treating instance domains as a 1-dimensional
vector indexed by an almost 32-bit instance identifier (2^31-1 is
special, aka PM_IN_NULL, so the PMDA gets the instances 0 thru
2^31-2 to play with) and allowing up to 256 PMDAs each with up to
4194304 (2^22) independent instance domains.
The limit on the size of a "string" value that can be returned from
a PMDA for a single metric-instance pair is 16777215 (2^24-1)
bytes.
In practice we've alwys been able to accommodate the apparent need
for a multi-dimensional instance domain by either:
(a) casting one of the dimensions into the namespace, i.e. enumerate
named metrics for each row or column in one of the dimensions,
e.g.
table.row1 ["col1"] val
table.row1 ["col2"] val
...
table.row2 ["col1"] val
table.row2 ["col2"] val
...
and/or
(b) encoding a set of values for one of the dimensions in a string
value associated with one of the other dimensions, e.g.
table ["row1"] "col1-val:col2-val:..."
table ["row2"] "col1-val:col2-val:..."
...
So we have definitely considered this capability (many times over),
but so far resisted the monumental changes that embracing this
would entail.
Counter-arguments, as always, would be most welcome.
|