Hi,
On 2015-10-30 08:37, Nathan Scott wrote:
> ----- Original Message -----
>
> As we discussed on IRC, that is at a much lower level than is relevant at
> this stage of evolution of a generic reporting API - if/when another new
> fetchgroup API comes along, it will be required to sit below these higher
> level APIs to prove its worth. So currently I'm more concerned about the
> high level API - pcp.pmcc is the start of one such API, but we don't have
> to use that if it doesn't suit... we'll see.
yeah, I'm also also taking the wait&see approach on that front, I think
finalizing pmrep will also help a bit to see potential client side
requirements for such an API.
> The 'cc' in pmcc stands for "Convenience Classes" - when current pmcc does
> not suffice (and it definitely will not - doesn't try to do this level of
> reporting granularity) we should be thinking if/how we can extend it with
> classes more convenient for generic reporting scripts.
If we find consensus and nail down the configuration file / metricspec
details, that might be the other side where generalization would be
possible.
> Do you have a public git repo somewhere? (github?) Would it make life any
> easier if you had a git.pcp.io account for your pcp tree? (if so, please
> send thru your public ssh key if so & I'll set that up)
That would probably be helpful, me operating on my own private tree is
not optimal although so far probably hasn't been an issue.
> There's no rush to merge, plenty of time in the current release window. The
> best next step will be writing up test cases for all of the existing export
> functionality that you have working, so that we don't go backward from there.
>
> We will want man pages and so on too before merging, but that can come later
> - automated testing of the currently-working-stuff would be most helpful at
> this stage.
I've now written a complete pmrep.1 man page and fixed few issues found
during the process and added few new features while at it. I changed the
"compact" metric spec to allow defining instances to be reported (not
yet implemented though) so that the order is roughly
"most-significant-modifier-first".
Unless there are any suggestions I'm starting to think that the user
interface should be considered stable in that sense that any possible
changes should be done without changing the current options and syntax
if at all possible.
The only thing might be the default time format used. It's currently
%H:%M:%S but timestamps with microsecond precision would be somewhat
tempting (see my earlier email wrt the archive records and pmval
reporting being in a sense "off-sync"). I also considered using a locale
specific default (like %X) but perhaps that's not a good default after all.
These and other ideas / possible to-do items are still listed on the top
of the file.
https://myllynen.fedorapeople.org/pmrep.py
https://myllynen.fedorapeople.org/pmrep.conf
https://myllynen.fedorapeople.org/pmrep.1
> This doesn't have to be in pcp/qa format yet - I can help with that later.
> Just a series of commands to run, and the expected output. Maybe focus on
> the csv and basic stdout reporting before Zabbix, since they're simpler to
> test - and try to get coverage of how pmrep handles all the different types/
> units/semantics/... of metrics, the different types of formatting, scaling,
> etc.
I enhanced the Zabbix reporting so that it can send metrics in batches
(in the spirit of zabbix_sender(8)) and it seems to work reliably. The
downside on that front is the the zbxsend upstream hasn't responded to
my zbxsend / Python 3 patches. One possibility might be to provide
pcpzbxsend which would essentially be zbxsend+fixes but I'm not sure
would this be optimal. For the time being I've added a note to the man
page the Zabbix output target being experimental.
> Use pmdasample metrics - that gives us a good variety of data, behaviours,
> whacky instance domains, metric types, etc - and its cross platform (which
> the kernel metrics tend not to be - so they're sometimes less suitable for
> this kind of regression testing).
>
> I'm away until middle of next week, but if you have some extensive testing
> in place when I'm back, I'll work on a full review, building pcp/qa tests,
> then we can modularise, document, and merge it.
The next step for me is the write the pmrep.conf.5 page, hoping to
finish it this or next week.
Thanks,
--
Marko Myllynen
|