pcp
[Top] [All Lists]

Re: [pcp] Checking PCP archives - RFC

To: Ken McDonell <kenj@xxxxxxxxxxxxxxxx>
Subject: Re: [pcp] Checking PCP archives - RFC
From: Nathan Scott <nathans@xxxxxxxxxx>
Date: Tue, 21 May 2013 19:02:30 -0400 (EDT)
Cc: PCP Mailing List <pcp@xxxxxxxxxxx>
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <519AC94B.9020904@xxxxxxxxxxxxxxxx>
References: <519AC94B.9020904@xxxxxxxxxxxxxxxx>
Reply-to: Nathan Scott <nathans@xxxxxxxxxx>
Thread-index: vammcWUJSXRUxWVja536Y+JyfoTxog==
Thread-topic: Checking PCP archives - RFC
Hi Ken,

----- Original Message -----
> The attached document spells out the case for pmlogcheck.
> 
> I'd appreciate feedback.

I think missing is discussion of the existing pmlogcheck(1) and how a
new tool will supplant that guy, presumably keeping the existing code
as a final pass?  pmloglabel also could get a mention - perhaps it'll
become a wrapper tool, since I'd expect its label checks would become
part of the bigger picture here.

pmlogcheck uses the PMAPI in non-interp mode.  pmloglabel doesn't use
the PMAPI at all for I/O, IIRC, and I think is more the model you're
thinking here (does the file I/O directly to get labels).

> If we proceed with this, there are several work items for which
> volunteer effort would be most appreciated.
> 
> 1. expand the list of things to check for

loglabel is checking consistency across the multiple files - be worth
doing that still, esp. since it was once a bug that those managed to
get out of sync somehow.

> 2. write the code (perhaps I'll take this one!)

Yes please! :)  I'm looking forward to helping out; keen to learn more
about some of the more ancient aspects of this code (metadata format &
temporal index format & usage).

On a somewhat related discussion for another time - I'll throw it out
here just to get the hamster wheel spinning - I have had occasion to
wonder whether we could index other things too, so not only for quick
time-based lookups but also fast searching of other things like event
records containing specific parameters, parameters in a range, etc.
Big, big project - not sure if this can/should be shoe-horned into the
existing archive format - so, a discussion for some other time.  ;)

> 3. turn your evilness volume up to 11 and start creating corrupt
> archives to be added to the qa suite for pmlogcheck to feed on
> 
> It would be _really_ good if different people were involved in 1. and 3.

Do you imagine this tool might be able to *fix* corrupt archives too?
pmloglabel does, and that was extremely handy at the time.  Even if a
subset of the data was all that remained... can still be crucial if it
is the only record one has of what happened in the past.

cheers.

--
Nathan

<Prev in Thread] Current Thread [Next in Thread>