Hi David,
----- Original Message -----
> [...]
> I've scratched my head over this for a good bit of today, and I finally
> figured it out. RHEL6 has python 2.6. The json.load() function in python
> 2.6 doesn't support the 'object_pairs_hook' which the json pmda uses. I
> haven't found a fix or workaround yet.
Any thoughts on this one since then? (not sure it was communicated, but
end result of this was Dave needed to switch pmdajson off in EL6 builds).
Given this, and the other niggling issues we're seeing from the python
modules pmdajson uses, maybe we should start thinking outside the square.
An alternative way to tackle this class of issue (incl. not having json
pointer packages at all on some platforms, and so on) could be to provide
a lower-level implementation of json parsing & pointing in libpcp_pmda.
We've already seen need for non-python PMDAs (pmdaroot, pmdaelasticsearch)
to parse JSON and we already have a teensy embedded json parser in C (~300
LOC IIRC). Given the increasing prevalence of json instrumentation in the
wild, perhaps we should promote its use as a first class PMDA concept, and
make use of (new) underlying API services in pmdajson and other PMDAs.
( Obviously, we need short-term stop-gap fixes for the python packaging
issues for some of our builds - possibly embedding the jsonpointer code
into the python pmdajson code as Frank suggested, or something like that;
but I'm thinking of longer-term, cross-platform (where "cross" includes
RHEL5, RHEL6, as well as Mac OS X, Solaris, Windows, etc), and not-just-
for-python-code solutions here. )
So, in this model we might:
- move JSON parsing into libpcp_pmda or libpcp_http from pmdaroot/jsmn.c
and make the pmdaroot Docker code use the shared library code
- implement jsonpointers in C (the spec looks quite simple, so I'd not
expect alot of C code there? - I'd be v. interested in your take on
that, David, since you've worked with it alot now)
- provide some clean PMDA-appropriate C API above 'em. Maybe not just
"parse this arbitrary-sized memory blob" (probably too simple - e.g.
we may want to be able to handle streaming json from a socket?), and
possibly using jsonpointers only at the API level? - that's all that
pmdaroot would need from an API, anyway
- interface this new PCP library code to python too, of course; also
to perl, and oh also for pcp-pmda-cpp C++ PMDAs if Pauls keen on that
- switch pmdajson over to using these new interfaces, removing those
problem "jsonpointer", "json.load", "six" module/version dependencies
entirely.
Again, this is all longer-term planning, its not going to be a quick fix.
It would resolve all the current packaging problems though, and bring the
wonders of JSON instrumentation to all platforms we support PCP on (which
is clearly not going to happen with a python-only solution - some of the
platforms do not have python at all, others do not have jsonpointers at all
(and are in stages of their life, like RHEL5/6, where this will not happen).
Thoughts? We'd need to flesh out the API a whole lot more, I've not given
a whole lot of thought to that yet (suggestions welcome!), and we may want
to pick a more general purpose embedded C JSON library to start from.
Incidentally, this may help to resolve some of the other pmdajson worries
still in the back of my mind (which I still owe you some mail on, sorry
'bout the tardiness there - will follow up soon).
cheers.
--
Nathan
|