pcp
[Top] [All Lists]

Re: Multi-Volume Archive + Live Data Playback for PCP Client Tools

To: Nathan Scott <nathans@xxxxxxxxxx>
Subject: Re: Multi-Volume Archive + Live Data Playback for PCP Client Tools
From: "Frank Ch. Eigler" <fche@xxxxxxxxxx>
Date: Thu, 2 Oct 2014 21:55:12 -0400
Cc: Dave Brolley <brolley@xxxxxxxxxx>, pcp@xxxxxxxxxxx
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <1741011764.60812520.1412300213647.JavaMail.zimbra@xxxxxxxxxx>
References: <542C21AE.1010504@xxxxxxxxxx> <y0megurfxp7.fsf@xxxxxxxx> <542C4EC1.4040407@xxxxxxxxxx> <408521091.60123629.1412223728964.JavaMail.zimbra@xxxxxxxxxx> <y0m1tqqcym4.fsf@xxxxxxxx> <542D756B.8030906@xxxxxxxxxx> <1741011764.60812520.1412300213647.JavaMail.zimbra@xxxxxxxxxx>
User-agent: Mutt/1.4.2.2i
Hi -


> [...]  If by -something- you mean a daemon, I'm unconvinced.  I
> think an archive (written unbuffered as we now do) would make an
> excellent -something- (as in, "tail -f" on an archive) along with
> direct use of either pmcd or local-context DSOs for live data
> (optionally). [...]

I see what you mean.  There are two complications:

1- This requires the client to divine pmcd server connection data for
the real-time component from the archives (or vice versa).  Or else
force the a user to specify both?

2- This makes it impossible to log the real-time collected data
uniformly with the other historical data.  Data that the mystery
pmlogger happens to log would be available; data that the PMAPI client
requests from live pmcd would be available; but data that the PMAPI
client *recently requested* wouldn't be.  If OTOH a pmlogger-intimate
server were the intermediary, then it can arrange for the
recently-requested data to be logged (alongside whatever static
configuration it had).  That way, when pmchart refreshes recent data,
the really old, the recent, and the current can all come from the same
place.  (The smarter pmlogger could even speculatively log more data,
in anticipation of future requests.)
   

> [...]  And there was other issues with that approach, like pmlogger
> having to duplicate all of the pmcd client-facing functionality (and
> more) - so, namespace/pmDesc/auth/PMID/value lookups, etc - its alot
> of code.

Sure, luckily that code is already written; just needs to be librarified.


> And another "hop" (synchronous round trip for every PDU) 

True (unless perhaps the new pmlogger can anticipate).


> whereas a direct pmcd connection is clearly available and simpler.

... assuming a direct connection -is- available!  It may not be.
Whatever pmlogger process is writing to the grand-unified archives
(consider archives coming via NFS from some faraway box) may have a
way to get to its pmcd that the pmapi client doesn't, whether via
special authentication, or network topology differences.  (IOW, 
as we still suffer here and there from the contrary assumption,
archive-label.hostname != usable PM_CONTEXT_HOST parameter.)


- FChE

<Prev in Thread] Current Thread [Next in Thread>