Hi,
On 2016-04-26 21:39, David Smith wrote:
> On 04/26/2016 01:30 AM, Marko Myllynen wrote:
>> On 2016-04-25 19:58, David Smith wrote:
>>> On 04/15/2016 06:12 AM, Marko Myllynen wrote:
>>>> On 2016-04-12 04:53, Nathan Scott wrote:
>>>>> ----- Original Message -----
>>>
>>>> Oh, well, I think I was being overly optimistic here.. But perhaps (or
>>>> even hopefully) some of the issues I'm listing below are due to my
>>>> misunderstanding (see the previous README patch - the current README
>>>> wasn't enough even to get a basic example working):
>>>>
>>>> * (When *testing* metadata, deleting /var/lib/pcp/config/pmda/137*
>>>> before each round may help to prevent some mysterious error messages)
>>>
>>> Without seeing the "mysterious error messages", I'm afraid I don't
>>> really know what the JSON PMDA was doing here.
>>
>> Actually, I was a bit confused what was going on but nevertheless there
>> are few surprising things. With metadata.json and data.json looking like:
>>
>> {
>> "prefix": "jsontest",
>> "metrics": [
>> {
>> "name": "a_b_c",
>> "pointer": "/a_b_c",
>> "type": "integer"
>> },
>> {
>> "name": "a_d_e",
>> "pointer": "/a_d_e",
>> "type": "string"
>> },
>> {
>> "name": "a_b_f",
>> "pointer": "/a_b_f",
>> "type": "integer"
>> }
>> ]
>> }
>>
>> {
>> "a_b_c": 1,
>> "a_d_e": "2",
>> "a_b_f": 3
>> }
>>
>> First clear the cache and install the PMDA to see all is ok. Then change
>> 1 to "1" in data.json, confusingly there's error printed for both a_b_c
>> and a_b_f (I'd expect error at most for a_b_c). Now revert "1" back to 1
>> and then verify things are ok again. Then change 3 to "3" and see how
>> the PMDA crashes.
>>
>> Of course, the user is responsible for providing correctly formatted
>> data but when testing (possibly with hundreds of metrics) such
>> inconsistencies in error cases can be a bit confusing.
>
> The PMDA certainly shouldn't crash there, that's a bug.
>
> As far as "providing correctly formatted data" goes, I'd expect that for
> most cases of hundreds of metrics, you wouldn't write the metadata by
> hand, but generate it from a script (as the generate_ceph_metadata
> script does).
Sure, nobody hopefully writes down hundreds of definitions/values by
hand. But when developing/testing any error slipping in might be hard to
find (although still possible).
>>>> * In the non-array case metrics a.b.c and a.d.e work but a.b.f is
>>>> silently omitted (so using a_b_c seems currently the only option)
>>>
>>> I'm afraid you lost me here. Can you explain a bit more?
>>
>> The above metrics still work if you do s,_,.,g but in a simalar simple
>> array case the PMDA log is being filled with errors and tracebacks.
>
> Without seeing the array case I couldn't guess at what is going on.
Ok, here are example files:
{
"prefix": "json",
"metrics": [
{
"name": "test",
"pointer": "/test",
"type": "array",
"index": "/__id",
"metrics": [
{
"name": "a.b.c",
"pointer": "/a.b.c",
"type": "integer"
},
{
"name": "a.d.e",
"pointer": "/a.d.e",
"type": "string"
},
{
"name": "a.b.f",
"pointer": "/a.b.f",
"type": "integer"
}
]
}
]
}
{
"test": [
{
"__id": "123",
"a.b.c": 1,
"a.d.e": "2",
"a.b.f": 3
}
]
}
With these you should see a failure but if you then change . to _
everywhere, everything starts to work ok.
>>>> * metadata.json updates are not picked up during JSON PMDA lifetime
>>>
>>> Yes, that is true. As far as I know, that's fairly standard for PMDAs,
>>> especially python ones.
>>
>> Ok, Nathan mentioned earlier that the Python PMDA API was extended to
>> allow adding metrics dynamically so if metadata.json updates are not
>> picked up then on the fly then perhaps he meant something else.
>
> OK, I looked into this one a bit more. I was a bit mistaken.
>
> If you add a new metadata source, the JSON PMDA will pick up the new
> source. If you change an existing source during the PMDAs lifetime, it
> will not notice the change. You'd have to restart the PMDA for it to
> reread the metadata.
Hmm, a new metadata source is added in pmdas/json/config.json, right? I
don't see the new metrics to appear in that case. Or is there another
mechanism to add a new source?
Thanks,
--
Marko Myllynen
|