pcp
[Top] [All Lists]

Re: Slow pmdapapi fetching

To: Nathan Scott <nathans@xxxxxxxxxx>
Subject: Re: Slow pmdapapi fetching
From: fche@xxxxxxxxxx (Frank Ch. Eigler)
Date: Fri, 23 Jan 2015 21:28:49 -0500
Cc: PCP <pcp@xxxxxxxxxxx>
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <765513326.15612778.1421976592499.JavaMail.zimbra@xxxxxxxxxx> (Nathan Scott's message of "Thu, 22 Jan 2015 20:29:52 -0500 (EST)")
References: <258293690.15607144.1421974332709.JavaMail.zimbra@xxxxxxxxxx> <765513326.15612778.1421976592499.JavaMail.zimbra@xxxxxxxxxx>
User-agent: Gnus/5.1008 (Gnus v5.10.8) Emacs/21.4 (gnu/linux)
nathans wrote:

> [...]  I noticed pmdapapi timed out for me on fetch (the old pmcd
> terminating connection to slow-responding PMDA after 5secs chestnut)
> [...]

Please see on pcpfans.git:

commit 07350e06314c39bec16adc502a1d2ba37e0dfd40 (HEAD, origin/fche/papi, 
fche/papi)
Author: Frank Ch. Eigler <fche@xxxxxxxxxx>
Date:   Fri Jan 23 21:19:59 2015 -0500

    papi pmda: add papi-refresh batching
    
    It has been reported that in some cases, a "pminfo -f papi" operation
    (causing auto-enable of all avilable counters) can take a noticeable
    amount of time.  This was due to incremental PAPI-level regeneration
    for each new added metric/counter.  That permitted precise errors, but
    costs time.
    
    This patch introduces a "papi.control.batch" parameter, which sets a
    limit for the maximum number of pmid's per fetch that permits the old
    behavior.  Abvove that limit, incremental PAPI regenerations are
    temporarily suppressed until the end of the fetch operation, so as to
    have one big PAPI bang.
    
    The default value (10) makes "pminfo -f papi" take <1 second.  Setting
    it to a large number (100s) can let it take up to multiple seconds,
    depending on machine load, perhaps even triggering pmcd's "timeout,
    you're gone!" treatment of the pmda.  Testing this aspect (stressing
    papi to timeout-failure) has not been included, but otherwise basic
    qa smoke-testing is present.
    
    In passing, the papi.c file is retabified, and an earlier pmUnits
    error introduced in commit ecfacf3ff for papi.control.auto_enable is
    corrected.

<Prev in Thread] Current Thread [Next in Thread>