It was a while ago, but IIRC there is no serial polling; the cluster
nodes register themselves with the PMDA on the head node, and then
periodically send a pmResult. The aggregating PMDA running on the head
node has a modified main loop with a select mask for the pmcd request
file descriptor as well as for every registered cluster node. The pmcd
connection is given priority if it's ready, and the cluster nodes are
processed based on who's ready to send data in ascending fd order. I
guess that might explain missing metrics for some cluster nodes (if
they stop sending for whatever reason).
Jeff, since the code is already GPL, perhaps post the source somewhere
and we can check it out.
Cheers
-- Mark
On Tue, Sep 6, 2016 at 9:57 AM, Nathan Scott <nathans@xxxxxxxxxx> wrote:
> Hi Jeff,
>
> ----- Original Message -----
>> > This is the daemon that aggregates indoms for per-cluster-node CPU
>> > data on the head node, so
>> [...]
>> See the emails from 11 August on Debugging sigpipe in pmda.
>>
>> But the real problem is that although pmclusterd exposes some 100 metrics or
>> so but only 20 of them are actually able to be fetched.
>>
>
> I expect the problem will be due to latency in the polling of remote cluster
> nodes, which IIRC is done in a serial fashion (one node after the other IOW)
> so one slow-reponding node will affect timeliness of all values?
>
> A design which did the remote fetching in parallel would be better suited,
> if so. You could go with a model where multiple processes fetch then write
> metrics using MMV(5) format - see also mmv_stats_init(3) - so a new PMDA may
> not be needed at all.
>
> cheers.
>
> --
> Nathan
|