pcp
[Top] [All Lists]

Re: Multi-Volume Archive + Live Data Playback for PCP Client Tools

To: Dave Brolley <brolley@xxxxxxxxxx>, "Frank Ch. Eigler" <fche@xxxxxxxxxx>
Subject: Re: Multi-Volume Archive + Live Data Playback for PCP Client Tools
From: Nathan Scott <nathans@xxxxxxxxxx>
Date: Thu, 2 Oct 2014 21:36:53 -0400 (EDT)
Cc: pcp@xxxxxxxxxxx
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <542D756B.8030906@xxxxxxxxxx>
References: <542C21AE.1010504@xxxxxxxxxx> <y0megurfxp7.fsf@xxxxxxxx> <542C4EC1.4040407@xxxxxxxxxx> <408521091.60123629.1412223728964.JavaMail.zimbra@xxxxxxxxxx> <y0m1tqqcym4.fsf@xxxxxxxx> <542D756B.8030906@xxxxxxxxxx>
Reply-to: Nathan Scott <nathans@xxxxxxxxxx>
Thread-index: s/gUPKosUu6ewwsmP7xBvtf6Zlw0kg==
Thread-topic: Multi-Volume Archive + Live Data Playback for PCP Client Tools

----- Original Message -----
> >> [...]
> >> I didn't follow the need to talk to pmlogger - what was the rationale
> >> there?
> > In the grand unified world, there will be a need to talk to
> > -something- that has both historical data and live data.  Making

If by -something- you mean a daemon, I'm unconvinced.  I think an
archive (written unbuffered as we now do) would make an excellent
-something- (as in, "tail -f" on an archive) along with direct use
of either pmcd or local-context DSOs for live data (optionally).

> > pmlogger into a network server would make it possible to have libpcp
> > engage in conversation with only a single thing.  pmlogger already
> > knows about its own archives as well as the pmcd it's pulling from.

Ah I remember this line of thinking now.  As you say, it doesn't help
for non-actively-written data, so is not as big a win as it initially
seemed.

And there was other issues with that approach, like pmlogger having to
duplicate all of the pmcd client-facing functionality (and more) - so,
namespace/pmDesc/auth/PMID/value lookups, etc - its alot of code.  And
another "hop" (synchronous round trip for every PDU) whereas a direct
pmcd connection is clearly available and simpler.

> Yes, that was the idea behind this suggestion. This would only be
> required for archive-transitioning-to-live situations.

Yeah, this is the case I was curious about the rationale for - there
doesn't seem to be an explanation so far as to why a libpcp "tail -f"
mode would be insufficient for the actively-written data.

> Historical-only,
> which is what I will address initially need not use a pmlogger and, in
> fact, a pmlogger need not even be running for this case.

Yep, understood.

BTW, note that the unified context concept requires *zero* daemons (live
data is optional and/or local context can be used).  This is important,
and a major design feature with stacking multiple contexts.  Which could
also be used to simplify and help preserve backward-compat for you here,
Dave, I think - building up using multiple archive contexts, and passing
back a handle to the top-level one from pmNewContext(3)?  Maybe.

> It just seemed to me that the archive-to-live case implied the existence
> of one or more running pmloggers and that each was already managing
> everything needed to make the transition.

I'm not seeing that as implied.  To my mind it implies the existence of
pmcd (well, actually, not even that - one could live-transition between
a PM_CONTEXT_LOCAL context and an archive) but to me it doesn't imply a
running pmlogger process.

I think the insertion of pmlogger here (or any other new daemon) would
vastly complicate an already complicated situation, and something that
we should avoid if we can.

cheers.

--
Nathan

<Prev in Thread] Current Thread [Next in Thread>