Developers meeting summary, 29/02/2012

[Nathan Scott <nathans@xxxxxxxxxx>]

Hi all,

Today we had the first of a couple of PCP developers meetings for
this week, which we have semi-regularly now.  Based on how easily
we filled an hour-and-a-half (until we had to give up the meeting
room to others), we should probably be having these more often!

We'll finish up tomorrow morning with the second meeting (with a
much bigger agenda!), and I'll send out a summary of that one too.
If others are interested in calling in, please let me know & it can be
arranged.


= Summary of Networking Working Group Meeting
= 29 Feb 2012, @Aconex Melbourne

Attendees:
Hamish Coleman, Mark Goodwin, Max Matveev, Nathan Scott, Ryan Doyle

Topics:
1. Started with a demo of the PCP protocol dissector for
Wireshark that Ryan has written, and recently begun jumping
through the hoops to get accepted.  Decoding working nicely
for all pmcd/client PDUs, some issues discussed:
   - Print PMIDs in dotted form, in the summary, rather
     than as decimal integer (broken out domain/cluster/item
     is available by drilling down), but at top level would
     be great to split apart too.
   - pmproxy protocol header not supported.  This would need
     to introduce new use of "conversations" concept (part of
     the WireShark APIs) which complicates things a bit, so
     was left out for first coding pass.
   - Using conversations would also allow us to remember name
     PDUs and possibly display metric names alongside PMIDs
     once that part of the PDU exchange has happened. Another
     level of complexity again, good chance not worth going
     to that length.
   - Decoding is fairly deep - most of the result structure
     is decoded (for which Ryan gets a medal!).  Event record
     metric type is not yet decoded yet though (decodes the
     type and reports it correctly, but not the body).
   - Generally, looking good and so far just relatively simple
     changes requested from initial upstream review.
     https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=6874

2. Moved on to discussion of issues Max has encountered while
working through the NFS client stats PMDA.  Issues around:
   - How to handle the instance domain, which is per-mounted
     filesystem but sometimes (when kernel decides to share
     struct superblock for a client NFS mount - same server,
     and same mount options) the instances will share the same
     values.  This is unexpected from a users POV, since the
     I/O went to one mount point or the other, yet both update.
   - Options include having a single instance for these shared
     mounts (assuming correct identification possible), using
     just whichever mount point is observed first as external
     instance name.  But, would lead to even more confusing
     behaviour if that mount point goes away, but the other
     remains - stats then reported for an unmounted path.
   - For further details, see Max's promised mail.
   - Would be helped by extension to the Perl PCP::PMDA module
     to allow PMDAs to call the pmdaCache family of routines
     (long overdue on my part).

3. Lastly we discussed where the SNMP PMDA was up to, with a
specific focus on the concept of transparent proxying to make
both client tool experience and PMDA implementation a little
more sane.
   - Discussed Ken's points about the host context being a
     first order item not to be trifled with, that archives
     should remain for only one host, etc.
   - All agreed.  Issue though is that what we're thinking here
     is trying to do a better job at that, in one area where it
     is causing us to shoe-horn the "host" concept into places
     it's not meant to go (instance domain or metric name).
   - Strawman put forward: pass hostname used in pmNewContext
     along with credentials PDU to pmcd, which can then make it
     available to PMDAs as well (this would piggyback onto any
     security extensions, likely to pass username, keys, etc on
     to PMCD/PMDAs - pcp4 stuff) so that the PMDA can use host
     name that individual clients are interested in to tailor
     the namespace and values made visible, per connection.
     DNS CNAMEs or other multiple name/address alias convention
     can then be used to allow PMDAs transparently (to clients)
     to tailor exported information.  Advantage would seem to
     be its simplicity, as it doesn't require modification to
     other PMDAs and uses existing/planned mechanisms.
   - Discussion moved on to whether this functionality could
     instead be handled through introducing a pmproxy plugin
     model, rather than using pmcd/pmda interfaces.  The idea
     would then be that pmproxy would talk PCP protocol to
     clients, and each plugin would talk SNMP (or any of many
     other protocols Hamish points out he'd also like to take
     on down the track) out the back.
   - In the end, seemed like we'd need most of the pmcd/pmda
     interface to be replicated, so not clear if that approach
     made sense in practice.
   - Mark very keen on the idea that we go further and make
     libpcp able to switch protocols, directly, in the client
     tools.  The idea would be to provide PMAPI above, and if
     initial PCP socket connection failed, fall back to SNMP,
     else next protocol, else next.  Nathan not a big fan. :)
     Some further discussion around a URL-like syntax for host,
     protocol and port specification (pcp://hostname:port, and
     snmp://hostname:port).  Still not a big fan - seems waaay
     to invasive, not transparent to clients at all, and puts
     alot of complexity in libpcp.  IMO.  :)

For the guys who were there, if I've missed anything (or too
heavily biased something with my own opinions!), please add
your thoughts/notes to this thread - thanks!

cheers.

--
Nathan