Hi!
Occasionally there's been some discussion on- and off-list how possible
PCP / Zabbix integration could look like. I investigated different
integration options in more detail and then implemented most of them.
The integration options I came up are all below but as you'll see some
of them can barely be labeled "integration". However, by listing them
all here we should have pretty complete picture of available options.
The impatient will skip directly to the items 5) and 6) to see how to
make the default 1000+ PCP perf metrics quickly available to Zabbix.
1) Collect metrics one by one on monitored hosts (PCP->Zabbix)
This can be done with user parameters in Zabbix agent configuration.
https://www.zabbix.com/documentation/3.0/manual/config/items/userparameters
As the page describes, it's one command per metric and basically we
could do something like the below for each wanted metric in the Zabbix
agent configuration file:
UserParameter=pcp.mem.util.free,pmprobe -v mem.util.free|cut -d' ' -f3
While this method uses a PCP client to fetch a metric value, I think
it'd be a bit of exaggeration to label this approach as "integration."
2) Collect metrics one by one from the Zabbix server (PCP->Zabbix)
This can be done with external checks executed on Zabbix server.
https://www.zabbix.com/documentation/3.0/manual/config/items/itemtypes/external
Here a script is executed on the Zabbix server for each monitored host,
for each monitored check. Although creating such scripts would be easy
(e.g. use a PCP client -h option with HOST.CONN), this method is not
really meant for retrieving individual performance metrics from each
monitored hosts, the overhead is simply too high and I don't think
this option warrants much of further discussion.
3) Collect host metrics from Zabbix agent to PCP (Zabbix->PCP)
This can be done with a PCP Zabbix Agent PMDA.
https://www.zabbix.com/documentation/3.0/manual/config/items/itemtypes/zabbix_agent
https://www.zabbix.com/documentation/3.0/manual/appendix/items/supported_by_platform
https://www.zabbix.com/documentation/3.0/manual/appendix/items/activepassive
It is possible to retrieve any items supported by a Zabbix agent with
zabbix_get(1) or directly over a socket, see the links under item 5.
This allows PCP to collect and store any Zabbix metrics. But looking at
the list of supported agent metrics, this isn't very tempting, basically
all performance related metrics are already supported natively by PCP
and the rest of the items would seem to be a better fit for Zabbix than
PCP.
However, in theory it might be possible that a Zabbix agent loadable
module (see item 6 for more details on modules), perhaps a proprietary
one, would provide metrics interesting also on the PCP side but not
available otherwise. I'm not aware of such modules and this is pretty
far-fetched but nevertheless I wrote a quick PCP Zabbix agent PMDA.
There's certain correlation between the ugliness of the code and its
uselessness so most people should probably completely ignore this one.
http://oss.sgi.com/pipermail/pcp/2015-October/008551.html
4) Collect all host metrics from Zabbix server to PCP (Zabbix->PCP)
This is purely theoretical but writing a PCP Zabbix Server PMDA should
be possible so that by doing SQL queries to the Zabbix server all the
data collected from monitored hosts to the Zabbix server could be then
fetched to PCP. But then again, what would be the point of this? I
can't think of any realistic general usage scenarios here so I haven't
implemented anything related.
5) Push a set of metrics from a host to the Zabbix server (PCP->Zabbix)
There's a Zabbix protocol to send (push) metrics to the Zabbix server.
https://www.zabbix.org/wiki/Docs/protocols/zabbix_sender/2.0
https://www.zabbix.org/wiki/Docs/protocols
https://pypi.python.org/pypi/zbxsend
What makes this interesting is that there's a Python module
implementing the protocol and the recently introduced pmrep utility,
which is written in Python, could then trivially expand to send any PCP
metrics to the Zabbix server. In fact, the addition of Zabbix support
to pmrep is so trivial that I'm pasting its main part below to
illustrate how little it takes to enable pushing all the PCP metrics
defined in pmrep.conf to Zabbix:
def write_zabbix(self, timestamp, values):
""" Write (send) metrics to a Zabbix server """
# Zabbix is always raw not rate
zbxmetrics = []
ts = float(self.ctstamp)
for i, metric in enumerate(self.metrics):
ins = 1 if self.insts[i][0][0] == PM_IN_NULL else len(self.insts[i][0])
for j in range(ins):
key = ZBXPRFX + metric
if self.insts[i][1][j]:
key += + "[" + str(self.insts[i][1][j]) + "]"
val = str(list(values[i])[j][2])
zbxmetrics.append(zbxsend.Metric(self.zabbix_host, key, val, ts))
zbxsend.send_to_zabbix(zbxmetrics, self.zabbix_server, self.zabbix_port)
http://oss.sgi.com/pipermail/pcp/2015-October/008553.html
6) Make PCP metrics available to Zabbix agent from module (PCP->Zabbix)
Loadable Zabbix agent modules (DSOs) can be used to extend the agent.
https://www.zabbix.com/documentation/3.0/manual/config/items/loadablemodules
The implementation is pretty straightforward and Zabbix agents which
load this module will then offer all the 1000+ PCP metrics to the Zabbix
server just like all the standard Zabbix items. For the implementation,
please see the follow-up message.
http://oss.sgi.com/pipermail/pcp/2015-October/008552.html
Thanks,
--
Marko Myllynen
|