pcp
[Top] [All Lists]

Re: [pcp] PCP libvirt PMDA

To: Martins Innus <minnus@xxxxxxxxxxx>, pcp developers <pcp@xxxxxxxxxxx>
Subject: Re: [pcp] PCP libvirt PMDA
From: Marko Myllynen <myllynen@xxxxxxxxxx>
Date: Wed, 13 Jul 2016 15:07:57 +0300
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <etPan.578509a6.57a05081.2ce@xxxxxxxxxxx>
Organization: Red Hat
References: <1fa58d82-ac73-7747-c58d-acf880bc2155@xxxxxxxxxx> <etPan.578509a6.57a05081.2ce@xxxxxxxxxxx>
Reply-to: Marko Myllynen <myllynen@xxxxxxxxxx>
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.2
Hi,

On 2016-07-12 18:15, Martins Innus wrote:
> Marko,
>       This is great!  I tried to run it, but I think my libvirt version may 
> be too old?
> 
> Under Centos 6:
> 
> $ libvirtd --version
> libvirtd (libvirt) 0.10.2

Seems that 0.10 is lacking some hugely helpful methods.

> $ pminfo -f libvirt
> 
> libvirt.dominfo.uuid
> Error: Resource temporarily unavailable

This should be always available.

> libvirt.dominfo.memory.max
> Error: Resource temporarily unavailable

This can be missing if maxMemory is not set for a domain.

> $ sudo more /var/log/pcp/pmcd/libvirt.log 
> Log for pmdalibvirt on srv-p23-14.cbls.ccr.buffalo.edu started Tue Jul 12 
> 11:02:58 2016
> 
> [Tue Jul 12 11:02:58] pmdalibvirt(120678) Info: Connected as root to 
> qemu:///system
> Traceback (most recent call last):
>   File "/var/lib/pcp/pmdas/libvirt/pmdalibvirt.python", line 356, in 
> libvirt_refresh
>     flags = libvirt.VIR_CONNECT_GET_ALL_DOMAINS_STATS_ACTIVE
> AttributeError: 'module' object has no attribute 
> âVIR_CONNECT_GET_ALL_DOMAINS_STATS_ACTIVE'
> 
> Do I need to configure something differently from the defaults or is this not 
> likely to work under Centos 6?

I've updated to code in my fedorapeople page to work with 0.10 API so
that it won't choke there anymore. But that doesn't help yet with the
missing metrics.

Below is a patch which collects the metrics available with 0.10 API. It
seems to work but as you see it's awfully ugly. The good thing, though,
is that it's rather well contained so those code paths are not hit with
Fedora / RHEL/Centos 7 / etc using libvirt 1.0 or newer.

Here's also a quick test script to allow to see what libvirt returns
to the caller if you want to verify that it's really working ok for you.

#!/usr/bin/env python

import libvirt
from lxml import etree

conn = libvirt.openReadOnly('qemu:///system')
doms = conn.listAllDomains(libvirt.VIR_CONNECT_LIST_DOMAINS_ACTIVE)

print(conn.domainListGetStats(doms, libvirt.VIR_DOMAIN_STATS_BLOCK | 
libvirt.VIR_CONNECT_GET_ALL_DOMAINS_STATS_BACKING, 
libvirt.VIR_CONNECT_GET_ALL_DOMAINS_STATS_ACTIVE))
for dom in doms:
    doc = etree.fromstring(dom.XMLDesc(0))
    src = doc.xpath("/domain/devices/disk")[0].findall('source')[0]
    path = None
    for path in 'file', 'block', 'dir', 'network':
        try:
            key = src.keys().index(path)
            path = src.values()[key]
            break
        except:
            pass
    if not path:
        continue
    print(dom.blockStats(path))

And here's the patch - I warned you :)

--- a/pmdalibvirt.python
+++ b/pmdalibvirt.python
@@ -370,6 +370,15 @@
                 if flags is not None:
                     stats = libvirt.VIR_DOMAIN_STATS_CPU_TOTAL
                     self.vm_cpustats_res = 
self.conn.domainListGetStats(self.doms, stats, flags)
+                else:
+                    for dom in self.doms:
+                        res = dom.getCPUStats(True, 0)[0]
+                        newres = {}
+                        for key in res:
+                            k = key.replace("_time", "")
+                            k = k.replace("cpu", "time")
+                            newres['cpu.' + k] = res[key]
+                        self.vm_cpustats_res.append([dom, newres])
             except libvirt.libvirtError as error:
                 self.log("Failed to get domain cpu stats: %s" % error)
             return
@@ -409,6 +418,36 @@
                 if flags is not None:
                     stats = libvirt.VIR_DOMAIN_STATS_BLOCK | 
libvirt.VIR_CONNECT_GET_ALL_DOMAINS_STATS_BACKING
                     self.vm_blockstats_res = 
self.conn.domainListGetStats(self.doms, stats, flags)
+                else:
+                    for dom in self.doms:
+                        doc = etree.fromstring(dom.XMLDesc(0))
+                        count = len(doc.xpath("/domain/devices/disk"))
+                        newres = {}
+                        newres['block.count'] = count
+                        for nr in range(count):
+                            src = 
doc.xpath("/domain/devices/disk")[nr].findall('source')[0]
+                            path = None
+                            for path in 'file', 'block', 'dir', 'network':
+                                try:
+                                    key = src.keys().index(path)
+                                    path = src.values()[key]
+                                    break
+                                except:
+                                    pass
+                            if not path:
+                                continue
+                            nrstr = str(nr)
+                            stats = dom.blockStats(path)
+                            for i in range(len(stats)):
+                                if i == 0:
+                                    newres['block.' + nrstr + '.rd.reqs'] = 
stats[i]
+                                elif i == 1:
+                                    newres['block.' + nrstr + '.rd.bytes'] = 
stats[i]
+                                elif i == 2:
+                                    newres['block.' + nrstr + '.wr.reqs'] = 
stats[i]
+                                elif i == 3:
+                                    newres['block.' + nrstr + '.wr.bytes'] = 
stats[i]
+                        self.vm_blockstats_res.append([dom, newres])
             except libvirt.libvirtError as error:
                 self.log("Failed to get domain block stats: %s" % error)
             return
@@ -419,6 +458,34 @@
                 if flags is not None:
                     stats = libvirt.VIR_DOMAIN_STATS_INTERFACE
                     self.vm_netstats_res = 
self.conn.domainListGetStats(self.doms, stats, flags)
+                else:
+                    for dom in self.doms:
+                        doc = etree.fromstring(dom.XMLDesc(0))
+                        count = len(doc.xpath("/domain/devices/interface"))
+                        newres = {}
+                        newres['net.count'] = count
+                        for nr in range(count):
+                            name = 
doc.xpath("/domain/devices/interface")[nr].findall('target')[0].values()[0]
+                            nrstr = str(nr)
+                            stats = dom.interfaceStats(name)
+                            for i in range(len(stats)):
+                                if i == 0:
+                                    newres['net.' + nrstr + '.rx.bytes'] = 
stats[i]
+                                elif i == 1:
+                                    newres['net.' + nrstr + '.rx.pkts'] = 
stats[i]
+                                elif i == 2:
+                                    newres['net.' + nrstr + '.rx.errs'] = 
stats[i]
+                                elif i == 3:
+                                    newres['net.' + nrstr + '.rx.drop'] = 
stats[i]
+                                elif i == 4:
+                                    newres['net.' + nrstr + '.tx.bytes'] = 
stats[i]
+                                elif i == 5:
+                                    newres['net.' + nrstr + '.tx.pkts'] = 
stats[i]
+                                elif i == 6:
+                                    newres['net.' + nrstr + '.tx.errs'] = 
stats[i]
+                                elif i == 7:
+                                    newres['net.' + nrstr + '.tx.drop'] = 
stats[i]
+                        self.vm_netstats_res.append([dom, newres])
             except libvirt.libvirtError as error:
                 self.log("Failed to get domain net stats: %s" % error)
             return

Cheers,

-- 
Marko Myllynen

<Prev in Thread] Current Thread [Next in Thread>