pcp
[Top] [All Lists]

Re: [pcp] pmdaCache concurrency and namespace issues

To: Mark Goodwin <mgoodwin@xxxxxxxxxx>
Subject: Re: [pcp] pmdaCache concurrency and namespace issues
From: Nathan Scott <nathans@xxxxxxxxxx>
Date: Fri, 1 May 2015 02:18:46 -0400 (EDT)
Cc: pcp <pcp@xxxxxxxxxxx>
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <55431755.5020408@xxxxxxxxxx>
References: <55431755.5020408@xxxxxxxxxx>
Reply-to: Nathan Scott <nathans@xxxxxxxxxx>
Thread-index: PtOh807NcfV3OIFk4ItxJR5nsa3n8Q==
Thread-topic: pmdaCache concurrency and namespace issues
Hi Mark,

----- Original Message -----
> The pmdaCache() functions read and write instance domain info
> to files in $PCP_VAR_DIR/config/pmda/* but I can't see any mutex
> between processes - I think it has checks for threaded apps,
> but separate processes could interfere with each other, e.g. two
> local context apps concurrently interleaving cache load and save
> operations. Do we need some kind of flock() protection here?

Maybe.

> Also, non-root users and users not in group 'pcp' can't write
> to these files when using a local context. Is that a problem?

Possibly - I guess a $HOME/.pcp/cache fallback could be added if
the system location is not writable?

> In addition - these functions may not be container safe. AIUI we

I think they are OK if used sensibly (i.e. shared across all of
the containers).

> switch namespaces temporarily but only to those name space(s)
> required for the metrics involved. e.g. fetching network metrics
> will use the net namespace for the named container ID, but the
> mnt namespace is not switched. Is that correct Nathan? If so,

Yep.

> the pmda refresh functions will save container specific instance
> domain info to the host's pmda cache files, which may then be
> completely wrong for the next app that loads that cache for
> the host or perhaps for a different container.

We share a single indom for all containers (and the host system) -
this means the above is not a problem, and also has other benefits
like not having linear increases in memory utilisation as more &
more containers are added (where all would have their own "eth0",
"lo0" duplicates, otherwise)

> Maybe we need to add the container id to each entry in each pmda
> cache? Would that work?

I don't think there's a problem needing to be solved here from the
containers POV - but the earlier local context questions, yeah, may
well be issues lurking there.

cheers.

--
Nathan

<Prev in Thread] Current Thread [Next in Thread>