pcp
[Top] [All Lists]

Re: [pcp] multithreading bottleneck: pdubuf.c

To: "Frank Ch. Eigler" <fche@xxxxxxxxxx>
Subject: Re: [pcp] multithreading bottleneck: pdubuf.c
From: Nathan Scott <nathans@xxxxxxxxxx>
Date: Sun, 1 Mar 2015 21:45:29 -0500 (EST)
Cc: pcp developers <pcp@xxxxxxxxxxx>
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <20150302015436.GB21203@xxxxxxxxxx>
References: <20150302015436.GB21203@xxxxxxxxxx>
Reply-to: Nathan Scott <nathans@xxxxxxxxxx>
Thread-index: 8WifaakpKhiEPQwv9UJVNgtGFhwFng==
Thread-topic: multithreading bottleneck: pdubuf.c

----- Original Message -----
> [...] a busy pmwebd shows a ginormous amount of
> traffic flowing through libpcp/src/pdubuf.c, many hundreds of
> thousands of requests per second.

The analysis lacks details as to whether this is a calling issue -
"hundreds of thousands of requests per second" is an obscene rate
of calls for any normal PCP application to be making - how many
web clients?  what sampling intervals?  archives/live?  ... what's
the distribution of PDU types in this pmwebd scenario?  Does this
match any valid real-world use or is it a test/benchmark?

Is this making use of pmwebapi_respond_metric_fetch()?  If so, you
may be optimising in the wrong place - it is extremely inefficient
in its use of PCP protocol/APIs (pmDesc/pmNameID request for every
PMID fetched, for every iteration of the fetch loop, i.e. on every
sample) - as we've discussed on several occasions.  Or perhaps its
some other, similar calling issue that'd benefit from some caching
in pmwebd to avoid making so many calls altogether.

cheers.

--
Nathan

<Prev in Thread] Current Thread [Next in Thread>