Hi David,
Hmm, some of my intentions here may have been misinterpreted a bit here,
let me try to clarify.
----- Original Message -----
> [...]
> For the record, we've got 3 (known) problems using the JSON PMDA with
> python 2.6.
>
> 1) "json.load" syntax error. This one is fairly minor, and might be
> fixed by switching to another one of the python json implementations or
> just removing the use of OrderedDicts. They were very helpful during
> development, since I was always processing JSON items in the order they
> were present in the input file, instead of in some semi-random order.
>
> 2) "six" module missing in RHEL6. This one is actually fairly minor. The
> "six" module is for python 2-to-3 compatibility. We could fix this by
> copying the fairly small parts of the six module we actually use into
> the python pmda. Another option would be to have 2 different versions of
> the python JSON pmda available, one for 2.6 and one for 2.7+.
>
> 3) "jsonpointer" module missing in RHEL6. This is the major one, and the
> reason I didn't bother working on 1) and 2). We could fix this at least
> 3 different ways I can think of: a) get the python-jsonpointer package
> into RHEL6, b) bundle the python-jsonpointer module with pcp for python
> 2.6, or c) get pcp and the python-jsonpointer module into DTS and ask RH
> customers who really want to use the JSON support use the DTS versions.
Thanks for the detailed run-down. This is the short-term, needed-right-now
sort of work I think need to be considered ASAP to get pmdajson into the
hands of those folks who can run/access recent-enough python packages.
My earlier mail was more along the lines of 'lets step back and think about
whether asking people to make all JSON accesses through a python pmdajson
(or rewriting existing PMDAs as pmdajson sources)' is going to be the right
thing to do for PCP users, in the long term. It's not clear to me, one way
or the other.
> This one is really a distro problem, anyone building from source
> themselves can obviously install anything they want.
Yep, definitely.
Keep in mind though that platforms like Mac OS X, Solaris, Windows - well,
even older Debians/Ubunutus/RHEL5 are places people want latest PCP, want to
access JSON data, and are not readily going to be able to use those above
solutions.
> As far as python goes, at some point you'll have to decide if python is
> a good way to write a PMDA or not. If it isn't, you should probably
> think about removing the python PMDA support. Ditto for perl.
>
Hmmm, well, not sure there - some people like perl, some like C, some python
- some access APIs might only be available in one language - I think people
will (and should be able to) choose to use the way to interface to PCP that
they prefer. Not sure we need to remove anything, or choose for them whether
one way is better - I do really like that we offer lots of choices for the
wide variety of environments and people we have using PCP.
> > Thoughts? We'd need to flesh out the API a whole lot more, I've not given
> > a whole lot of thought to that yet (suggestions welcome!), and we may want
> > to pick a more general purpose embedded C JSON library to start from.
>
> Wow, this is far outside the square. Resolving the current packaging
> problems seems like a problem that is quite doable in several different
> ways, and what you are describing above is several magnitudes of effort
> above fixing the packaging problems.
Yes and no. The problem is the packaging problems are unfixable for some
platforms/deployments ... hence, I want to think ahead for "next steps"
now that we've had early successes with the initial pmdajson work - just
trying to think where we *could* go to help those folks we are
(completely unintentionally) stranding a bit here.
> Putting JSON parsing functionality into libpcp_pmda or libpcp_http seems
> a bit odd to me. It doesn't seem like it fits in either, especially
> libpcp_pmda.
Yeah, it depends how far we go though - e.g. we could make it so easy to
write a JSON-consuming PMDA (by filling in fetch/instance/etc callbacks
automatically and so on) that maybe it would be needed in libpcp_pmda.
> I'm not sure I see the value in the short-to-medium term for rewriting
> everything in C/C++. You'd certainly have a better idea of the long term
> benefits.
Yeah, and this is mostly "thought experiment", "trawling for ideas" kind of
stuff at this stage; that may not have been clear in my earlier mail, sorry.
> If you did decide to go this route, I'd *certainly* pick a
> well-supported general purpose C JSON library to start from.
*nod*
> > Incidentally, this may help to resolve some of the other pmdajson worries
> > still in the back of my mind (which I still owe you some mail on, sorry
> > 'bout the tardiness there - will follow up soon).
>
> As far as I know all the worries that you've mentioned to me in the past
> have been addressed. If you've got some new ones, send them on and I'll
> look at them.
Oh, I was referring to my earlier comments about how there were some things
we should defer to discussing after initial merge of pmdajson code, so that
they didn't get in our way. Nothing really buggy (well, hmm, maybe sorta -
more design stuff, needing deep consideration) - but certainly nothing that
gave me pause, or stopped me from thinking pmdajson was a good approach to
be merging right now.
I'll send some notes through early next week, still a bit swamped under the
vacation backlog here.
cheers.
--
Nathan
|