pcp
[Top] [All Lists]

Re: pcp updates: new pcp-atop implementation

To: "Frank Ch. Eigler" <fche@xxxxxxxxxx>, Stan Cox <scox@xxxxxxxxxx>
Subject: Re: pcp updates: new pcp-atop implementation
From: Nathan Scott <nathans@xxxxxxxxxx>
Date: Fri, 29 May 2015 02:03:03 -0400 (EDT)
Cc: pcp <pcp@xxxxxxxxxxx>
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <y0ma8wpq23z.fsf@xxxxxxxx>
References: <32035133.6250974.1432707468770.JavaMail.zimbra@xxxxxxxxxx> <1044667991.6256856.1432708241762.JavaMail.zimbra@xxxxxxxxxx> <y0ma8wpq23z.fsf@xxxxxxxx>
Reply-to: Nathan Scott <nathans@xxxxxxxxxx>
Thread-index: Z3ks7uuUuCy7dQ3+nB6/Zu7NQz8U5g==
Thread-topic: pcp updates: new pcp-atop implementation
(Thanks to everyone who has sent me a private note about the new
atop - it is encouraging to hear the positive feedback!)

Frank Ch. Eigler <fche@xxxxxxxxxx> writes:
> > [...]
> OK, that answers my question about how this code is to be shipped.
> 

The thread I pointed you towards earlier clearly indicates Stan and
I planning to directly *replace* the current python code.  It also
discusses working with the upstream author, as in: "maybe we could
even begin to foster a relationship sharing fixes/features with the
original author." - there is no lack of clarity there.

Once again, for reference:
http://pcp.io/pipermail/pcp/2015-March/006867.html
http://pcp.io/pipermail/pcp/2015-March/006870.html

> Could you explain the motivation for forking atop (as contrasted with
> adding PMAPI extensions into upstream atop)?

I will try, though may prove difficult to explain some of the finer
points by email...

Firstly, it may appear from looking at the HEAD code in the updated
repo that simply "adding PMAPI extensions into upstream atop" was a
decision actually available to me at some point.  But realistically
it never was.  While the code before and after both look relatively
maintainable ... a smashing-together-of-the-two certainly would not
be ([*] <- see below for details, or delve deeply in the code).

Secondly, it is no more a fork than what we already have - a second
implementation of atop to maintain.  This new version is a big step
closer to a grand-unified version though.

Thirdly, over time we may be able to reduce the difference & attempt
to merge some/all of the PMAPI components upstream.  This isn't going
to be an overnight process though, and we have to (continue to) ship
a pmatop *now* (not to mention start extending it for containers).

I'm certain we have the best possible situation to move forward with
at this time.  I am also quite comfortable that the folks who are
actually maintaining PCP today will have no trouble with maintaining
this replacement code too - it's just as Stan envisaged when he said
"It would be a big improvement."  It definitely is.


[*] If you actually look at the code, specifically the delta between
the initial commit I made of Gerlof's code unchanged versus what will
be in the next PCP release, you will notice that the changes are very
intrusive in some areas.  In terms of merging back upstream, they are
prohibitively so.  There is no way around that - this is not fixable
by a sprinkling of #ifdefs.

In other areas, we have no PCP equivalent functionality at all (like
use of process accounting + netatop - these would need PMDAs & PMDAs
that duplicate other daemons in the upstream atop codebase no less).

In yet other places, the work to dovetail two models (PMAPI vs direct
kernel accesses) would be extremely awkward to support - so, unlikely
to be accepted upstream, and likely to be a source of bugs that PCP
developers like myself would then have to fix.


That said, there is much code that can be shared right away - I've
sent a first patch to Gerlof already, with several others planned, to
begin sharing fixes and to reduce the delta in those places where it
makes sense.

But that's future-planning stuff, and depends on a relationship being
built over time - not helpful to those of us maintaining this *right*
*now*.  And we needed a short-term solution, as Stan and I discussed
... so, here we are.

>  Separately, does this
> imply a pcp policy change that bundling third-party sources into the
> main pcp repo is henceforth considered acceptable?

This code certainly meets all the criteria we've applied in the past
for including outside code into PCP - just like the getopt code, the
getdate code, the jenkins hash code, and the several other instances
- all are very similar to this.  The code is well-written, mature, &
I've audited all of it carefully over the course of the porting work.

So, no, no change in the way we've operated to date is needed.

cheers.

--
Nathan

<Prev in Thread] Current Thread [Next in Thread>