On 02/19/2016 05:38 PM, Ken McDonell wrote:
...
collectl2pcp can take multiple input logs from collectl and create a
single PCP archive. The input logs are often not temporally
contiguous, i.e. there may be a time interval between one log and the
next.
Replaying the PCP log produces some unexpected results, and I suspect
this may be due to missing mark records. Is there any way to write out
a <mark> record in the PCP log, between processing input files? I
don't see any pmiFoo API function in LOGIMPORT for this ...?
maybe it's better to write individual PCP logs, and then merge them?
Mark,
If you're sure about the temporal discontinuity (a real gap between the end
of one archive and the start of the next archive in temporal order), then I
there are definitely temporal gaps between the collectl input files - that's
what triggered me to investigate the miss8ing mark records in the output
archive, along with some clearly wrong interpolated pmchart plots.
could see the case for adding pmiPutMark() or similar to the Perl LogImport
module (I don't think this is a libpcp candidate, so the implementation will
need to be entirely in the perl wrapper).
The code behind pmiPutMark() is a little tricky ... see putmark() in
pmlogger/src/callback.c
ok, thanks for the pointer. I've taken that pmlogegr code as an example and
added pmiPutMark() to libpcp_import, see attached patch. [with obviously
more work todo to add this to the perl binding, qa tests, docs, etc.]
On the other hand, if collectl2pcp can't be sure of the temporal semantics,
it would be better to process one archive at a time and then let
pmlogextract merge 'em all if you need a single output archive.
I tested the patch by using collectl2pcp on each inputfile, then used
pmlogextract to merge them all together. And then used the new collectl2pcp
to create a PCP archive from all the input files in one go. Then compared
with pmdumplog -m | grep '<mark>' - the results and <mark> timestamps
are in agreement.
So the options here are to either take the patch and extend libpcp_import,
or to rip out the multiple-input-file support in collectl2pcp, and
instead rely on pmlogmerge. Seems to me there are legit uses for pmiPutMark(),
so I'd rather go with the patch. Thoughts?
Regards
-- Mark
pmiPutMark.patch
Description: Text Data
|