On 04/09/2015 09:58 PM, Nathan Scott wrote:
>>> ... needs to be deterministic, else bugs - see mail re dmcache/dmthin a
>>> little earlier for more details. IOW, restarting/reconfiguring the PMDA
>>> needs to ensure the same IDs are generated for the same metrics/indoms.
>>
>> I couldn't find the email you were referring to, but I see the problem
>
> Oh, sorry, that was an obscure reference - this is the one I meant:
> http://www.pcp.io/pipermail/pcp/2015-March/006876.html
>
>> with keeping the IDs the same for the same metrics/indoms. I'm not sure
>> I can think of a good scheme to fix that problem.
>
> Nor I - unfortunately I don't think there's an easy answer here. :(
>
>> Do any of the other
>> PMDAs solve this problem for non-static metrics?
>
> No, there aren't really many in the same class - other PMDAs either encode
> fixed IDs (the vast majority), or do it poorly for small subsets of their
> names (like the percpu interrupts metrics - known bug) ... and those latter
> cases are hanging out for a real solution too. :)
>
> In the case of MMV which is probably conceptually closest to this PMDA, the
> numbering is pushed all the way out to the application, which encodes fixed
> identifiers.
>
>> We could use a crypto hash function,
>
> Yeah - something like that - have a look at src/libpcp_pmda/src/cache.c as
> thats how the instance cache number stability is achieved. Perhaps we can
> extend that with additional APIs to help us out here.
>
> If we start extending libpcp_pmda, we should also think about routines like
> generate_pcp_name() and your name() validity checker - the folks at Buffalo
> recently expressed need for those same interfaces for C PMDAs, so it would
> be great to have a common implementation.
>
>> but we've only got 22-bits to encode the cluster/item in.
>
> Yeah. Actually that's another little limit lurking here I guess - each JSON
> source can provide a max of 1024 metrics ("item" limits us, as cluster used
> for distinguishing JSON sources)? I've seen that passed sometimes with some
> large instrumented applications, but its infrequent. Just something else to
> keep in mind here I guess, maybe the end hash/numbering solution can seek a
> way to lift that restriction too, somehow.
I've looked into the idea of using a hash/crypto hash function, and I
just don't see a way to do it within the current api. 22-bits to encode
the cluster/item just isn't enough, especially if we want a unique hash
value.
For the JSON PMDA, we put each JSON source in its own cluster, so it can
support 4096 JSON sources, each with 1024 metrics.
If we don't want to break the current api/abi, the JSON PMDA could just
store its state (which JSON source maps to which cluster and which
metrics within a source map to which item) in a file when it quits, then
restore it at startup.
Is it safe to ever reuse a cluster/item?
--
David Smith
dsmith@xxxxxxxxxx
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)
|