----- Original Message -----
> Hi -
>
> > File:etc-pcp-pmlogger.tgz 8 KB
> > File:etc-pcp-pmmgr.tgz 1 KB
> > File:var-log-pcp-pmlogger.tgz 7501 KB
> > File:var-log-pcp-pmmgr.tgz 115240 KB
>
> The two pmlogger.log files identify the apples & oranges directly.
>
> pmmgr's pmlogconf:
> "Auto-generated by pmlogconf on: Friday 14 February 09:11:41 EST 2014"
> "254 metrics ... logged every 60 sec: 44340 bytes or 60.89 Mbytes/day"
>
> other pmlogconf:
> "Auto-generated by pmlogconf on: Mon Jul 22 12:48:51 EST 2013"
> "100 metrics ... logged every 60 sec: 4496 bytes or 6.17 Mbytes/day"
>
Yep very good point, interesting - those are not default log sizes I'd
expected (for either!). Ohhh, its found and enabled postgres logging!
And the other config is really out-dated.
Re-creating the pmlogger log conf, and same config generated now. I
should revisit the frequency with which pmlogger_daily auto-generates,
not clear that's working as planned.
OOC, any thoughts on the question of merging logs during the middle of
the day? Restarting pmcd appears to be an expensive operation with a
pmmgr in play without some strategy there? (I can restart pmcd/pmmgr
a whole lot more quickly than those logs can be merged)
And, yeah, wow - hmm, thats alot of logged data now, I wonder if we're
going overboard with some of the logging configs. :|
Yet another issue - the daily logs should be compressed after a few
days. It seems the daily script has stopped some of its functions?
Rotation is happening but cull &| compress appears to not be anymore.
Oh here we go... running daily script by hand, verbose mode reveals:
pmlogger_merge: Warning: archive "20140123.09.55" is empty and will be skipped
Input archives to be merged:
20140123.00.10
20140123.16.25
pmlogextract: Error: __pmLogRead[log 20140123.00.10]: Corrupted record in a PCP
archive log
pmlogextract: Error occurred at byte offset 2690960 into a file of 2691072
bytes.
The last record, and the remainder of this file will not be extracted.
Archive "20140123" not created.
pmlogger_merge: Directory: /var/log/pcp/pmlogger/verge
pmlogger_merge: Failed: pmlogextract 20140123.00.10 20140123.16.25 20140123
pmlogger_merge: Trying to continue, although output archive may be corrupted.
Output archive files:
pmlogger_merge: Error: file "20140123.meta" not created
pmlogger_merge: Error: file "20140123.index" not created
pmlogger_merge: Error: file "20140123.0" not created
Merged output archive 20140123 ...
pmdumplog: Cannot open archive "20140123": No such file or directory
Skip culling and compression ...
Hmm, file corruption. Oh man, that never happens! ;) (/me ducks and
runs). And an empty log file in the mix too.
In terms of the cull/compress though, the last line is key ... it
seems overly drastic - but appears to be working as planned, Ken?
Self-correction would be good, once the problem archives have all
scrolled past their use-by date?
BTW, the option to compress is also missing in pmmgr, I realise now;
another one for that list. Not easily implementable either, it would
seem, with one merge log - probably need libpcp to be able to do this
directly - bump log format, add code to do in-line compression...?
Or "hack" it, and inflate/deflate around the merges...? That defeats
the point of having the one continuous log handy though, I think, so
it really would have to be in libpcp. (a good feature anyway, IMO)
cheers.
--
Nathan
|