On Mon, Mar 14, 2016 at 04:38:26PM -0400, Nathan Scott wrote:
> Hi Alec,
>
> ----- Original Message -----
> > My question is: is it default for PMDAs to grab metrics for as many
> > nodes as possible, or only on the node that PCP is running on?
>
> The latter is definitely preferable and not only because of the additional
> latency, but also things like the hostname in PCP archives being incorrect
> and so on.
Interesting, completely forgot about that.
> Ideally we would have separate PCP collectors running on each
> system exporting just local metrics, and use PCP protocols (via the client
> tools) to address the need to extract metrics from multiple systems.
This is definitely the setup we want, just not the way the PMDA is
written.
>
> > There is
> > an easy way to patch the PMDA to fix our problem, but it would break the
> > behavior the PMDA has historically had. What is the best way to go about
> > fixing this?
>
> Can you describe this change a bit further?
Change one URL from "_nodes/stats" to "_nodes/_local/stats". Problem
gone.
> There are definitely ways we
> can tackle this, such as using different-named metrics for the local node
> case, renaming the "elasticsearch.nodes" metrics & indom (has good error
> handling characteristics for anyone using the old metrics).
I see that a configuration file PCP_PMDAS_DIR/elasticsearch/es.conf is
parsed by the PMDA. I was thinking that it would be possible to add a
new variable, something like $all_node_stats. It would default to false,
so only metrics from localhost would be fetched. It could be set true by
that configuration file.
Your solution is fine too; I'm new to PCP, so I will be glad to patch it
in whichever way fits best with the project. Is your solution just
leaving as is, but replicating the "elasticsearch.nodes" metric tree to
something like "elasticsearch.local_node" so old behavior is preserved
and the new behavior is added?
Thanks,
Alec
|