Hi Frank,
----- Original Message -----
>
> nathans wrote:
>
> > [...] I'm convinced that now is a good time to change the way we
> > manage PCP sources, while it's still early days in the PCP web &
> > middleware areas [...]
>
> I'm unconvinced that cleaving the project this way is helpful, and
> wonder what other members of the community think.
I've set this aside for a week so others could chime in with their
own opinions. Obviously there's not a whole lot of interest; I guess
people are expecting us to be just getting on with resolving it, one
way or another.
So, we need to move forward once more. I remain very much convinced
that separate trees will be a positive gain for us all, and will be
moving forward along that path shortly. I've considered your points
below - see my notes that follow - but at this time feel we must all
move on without getting bogged down in weeks of further argument.
I remain uncomfortable with maintaining those components that will be
in the new pcp-web-manager tree, as we've discussed. If this is not
something you want to take on being the primary maintainer for, then
that's a bit of a problem. You & I are the main people who've hacked
on all of that code (other than the hundreds of thousands of lines of
embedded javascript code, heh) - mainly you, see my next mail. In that
case, I'll continue on in a caretaker role until someone else (anyone?)
volunteers - just in the short term.
I do remain hopeful that you'll come around though. Some goals - like
the QA discussion we've been having - are immediately achieved using
this new tree.
I've found and fixed several problems in the web test (i.e. the artist
previously known as qa/660), but one or two issues remain unsolved -
the test doesn't pass for me on f20 even before the changes. I shall
follow up separately on those issues, tomorrow.
> > ----- Original Message -----
> >> > [...]
> >> > As discussed elsewhere before your vacation, I'm not comfortable
> >> > adding copies of 100s of 1000s of lines of javascript code and
> >> > images to core PCP from projects that we didn't author, nor have
> >> > plans/skills to maintain ourselves.
> >>
> >> I'm not comfortable either, but solutions that make everyone
> >> comfortable are not likely to exist soon.
> >
> > Agreed. I'm semi-comfortable with this being released separately to
> > core PCP, and not at all comfortable with any of it embedded in the
> > PCP git tree.
>
> Yet this proposed alternative scheme puts them into the same
> e.g. fedora distribution packages. It does not solve any hypothetical
> licensing concerns, or team workload, or bug triage procedures, or
> indeed who fixes what bugs.
Oh, it certainly does solve many issues - you are just thinking mainly of
Red Hat. It means I can continue pushing Debian packages to the archives
for core PCP (with my community debian.org hat on, I'll not be maintaining
the web tree code - others who are interested can take that on, & convince
themselves the code is fine if they wish, but I draw the line there).
The hypothetical licensing concerns of having encumbered code introduced
into core PCP are indeed addressed *for core PCP* since said code is not
introduced and only the web tree (or the jvm tree or ... any other future
tree building substantially on other peoples work) containing potentially
encumbered code would be at legal risk.
In terms of Fedora RPM builds, the use of %source would be unnecessary if
we had separate spec files. However, I know it's very useful for Red Hat
processes if we have a single spec for PCP, so that bit's more a practical
solution, and not my technically-preferred-solution.
>
> >> Would your discomfort be eased if this same non-embedding embedding
> >> were experimented with for purposes of the web applications? Namely,
> >> include src tarballs of graphite/grafana/etc. within pcp src.rpm's,
> >> and have these be untarred under /usr/share/pcp/jsdemos during the
> >> build/install phase?
> >
> > That's an improvement of sorts, but still only a stop-gap measure.
>
> And yet the proposal is to move pmwebd core code out too - a baby &
> bathwater overreaction.
On the contrary, it is deeply tied to the javascript code - see for
example the >2000 lines of custom C++ code to deal with Graphite alone.
I am sure other deep ties will develop over time with other javascript
code, either that we write or that's pulled in from elsewhere. Zabbix
is likely to need this treatment too, FWIW.
> > [...] The core PCP tree would be a poor place to be doing this IMO
> > - with its current build system, testing model not ideal for
> > web/javascript code, and so on. [...]
>
> Please substantiate this concern. It is difficult to see how the
> power of arbitrary shell scripts (qa/NNN) is not sufficient to wrap
> whatever mechanical webapp-testing widget might eventually be used.
I've not explained clearly there sorry, although if you have looked at
the other layered trees that Paul Colby is working on, it might have
become more clear. The goal is to allow the use of other "modern" ways
of testing, and not be tied to pcp/qa. See the related discussion of
pcp/qa: http://www.pcp.io/pipermail/pcp/2014-September/005679.html
> > 1. The class of problem that Ken raised here ...
> > http://www.pcp.io/pipermail/pcp/2014-September/005440.html is going
> > to become far worse for javascript and java builds. [...]
>
> I don't see the connection. Ken rightly points out that the
> build/test trees must be flexible as to prerequisites and older
> platforms. But they can/should be identically flexible, whether
> or not they are stored in the same git repository.
Perhaps I can explain more succinctly this way. We have many people
working in small parts of core PCP, however only three people actively
contribute to the full system testing required for upstream releases.
IMO, this is at least partly due to the nature of the pcp/qa testsuite
being large and complex already, but also because other people simply
do not want to do full system testing. I do not plan to add javascript
and java and any other large testing mechanisms into that mix, adding
to the full-system upstream QA burden already carried exclusively by
Ken, Dave and myself - we're stretched as-is, and as a result some of
us have greatly reduced time for new development work.
The best path to tackling this is to have focussed trees for complex
sub-system cases like web and middleware. The people choosing to take
on each of those subsystems can also take on the role of releasing and
testing for each area. And they can do it more efficiently too - each
can use testing methods that suit their own area best - web/middleware
developers from those subsystems can use the tools/suites they prefer,
and we wont end up with one lowest common denominator testsuite that is
maintained by just a few "lucky" souls.
> > 2. Already, the web package is approx double the size of core PCP
> > (2.9M vs 1.5M) [...] It contains 300+ more files than the core PCP
> > package [...]
>
> Is there a maximum limit for a core-pcp source distribution in bytes
> and file count? What resources are at risk of exhaustion by an extra
> few megabytes of sources or optional binaries?
There is certainly a point where we have to stop and rethink (and along
with all the other issues, yes, definitely we've passed that point for
the code in the pcp-web-manager tree - we will certainly pass it in the
middleware space rapidly too, based on my investigations there so far, so
I plan to start with a separate tree from day 1 there). I don't think
there's any specific numbers if that's what you're seriously after - its
more an application of common sense, and ensuring that the contents of
each tree is a good fit for those folks actively maintaining them.
I think I mentioned, several times now, I'm not comfortable continuing to
maintain the code in that tree. I hope you can respect that, just as I
have respected the right you have to the kind of implementation you chose.
I'll not be continuing an argument over weeks and weeks, however - I have
considered all of your points, and the lack of concern from anyone else,
and will be moving forward on this path.
So, thanks for the feedback; I'm a bit saddened you didn't like CMake as
an alternative build system (again, I'd encourage you to review pcolbys
work - its really cool IMO), but its your call so I've stuck with plain
autoconf/make for now in the new tree. Feel free to change it - one of
the many advantages of this approach is that the build/test/deploy system
there could be changed quite quickly now if you/others would like to.
I'm looking forward to handing over what I think is a neat separation
on this tree tree shortly (within pcp-3.10.0 timeframe), and continuing
on with my focus on core PCP. I wish all the best to whoever is keen to
take on responsibility for the new tree - hopefully it's you, Frank, I
think you'll do a good job.
cheers.
--
Nathan
|