pcp
[Top] [All Lists]

Re: [pcp] trouble querying containers

To: Paul V Novarese <paul.novarese@xxxxxxxxxx>
Subject: Re: [pcp] trouble querying containers
From: Mark Goodwin <mgoodwin@xxxxxxxxxx>
Date: Tue, 07 Jul 2015 15:06:36 +1000
Cc: pcp@xxxxxxxxxxx
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <CADFSn8By3tGpMNH3P3w_z=hEtaGqaH+JfLhN8aDqvvMw6mjW3w@xxxxxxxxxxxxxx>
References: <CADFSn8CZg_LXrwW9dHTt_+UAY51EhLXeAoWh5VBwkVtY4qiurA@xxxxxxxxxxxxxx> <5599DA06.1020305@xxxxxxxxxx> <CADFSn8By3tGpMNH3P3w_z=hEtaGqaH+JfLhN8aDqvvMw6mjW3w@xxxxxxxxxxxxxx>
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0
hmm. There was an issue a while back with selinux refusing to
allow bind to the root PMDA's socket. Can you retry with selinux
in permissive mode? (after editing /etc/selinux/config, you'll
probably need to systemctl restart pmcd).

If that doesn't fix it, then please edit /etc/pcp/pmcd/pmcd.conf
and append -DATTR to the line for pmdaroot, and then SIGHUP pmcd.
Then re-run the test and check /var/log/pcp/pmcd/root.log or
better, post it here. The root PMDA prints lots of debug info
concerning container name matching if you run it with -DATTR.

Cheers
-- Mark


On 07/06/2015 11:19 PM, Paul V Novarese wrote:
Hi Mark et al,

Same result using name or id.  Fetching containers.name <http://containers.name>
doesn't seem to work, so I guess it's not surprising that I can't query
individual metrics from a container.

pvn@gyarados /home/pvn> pcp verify --containers
pvn@gyarados /home/pvn> echo $?
0
pvn@gyarados /home/pvn> pminfo --fetch containers.name <http://containers.name>
containers.state.running

containers.name <http://containers.name>
No value(s) available!

containers.state.running
No value(s) available!

pvn@gyarados /home/pvn> docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED
STATUS              PORTS               NAMES
be8a4cce7e24        busybox             "/bin/sh"           9 hours ago
Up 9 hours                              distracted_mclean

pvn@gyarados /home/pvn> pcp
Performance Co-Pilot configuration on gyarados:

  platform: Linux gyarados 4.0.5-300.fc22.x86_64 #1 SMP Mon Jun 8 16:15:26 UTC
2015 x86_64
  hardware: 4 cpus, 4 disks, 1 node, 12011MB RAM
  timezone: CDT+5
  services: pmcd
      pmcd: Version 3.10.6-1, 7 agents
      pmda: root pmcd proc xfs linux mmv jbd2

pvn@gyarados /home/pvn> docker version
Client version: 1.7.0
Client API version: 1.19
Go version (client): go1.4.2
Git commit (client): 0baf609
OS/Arch (client): linux/amd64
Server version: 1.7.0
Server API version: 1.19
Go version (server): go1.4.2
Git commit (server): 0baf609
OS/Arch (server): linux/amd64

On Sun, Jul 5, 2015 at 8:29 PM, Mark Goodwin <mgoodwin@xxxxxxxxxx
<mailto:mgoodwin@xxxxxxxxxx>> wrote:

    On 07/04/2015 06:48 AM, Paul V Novarese wrote:

        hi container fans,

        I am trying to follow along with 
http://www.pcp.io/docs/lab.containers.html

        I built 3.10.6-1 from git after having trouble with 3.10.4-1 (in the 
latest
        fedora 22 repository)

        I am pretty sure I have gotten this working in the past but I don't 
seem to
        remember what I've done.  The pmdaroot daemon is running but pminfo,
        pmchart,
        etc all give me a container not found error.

        [root@gyarados ~]# docker ps
        CONTAINER ID        IMAGE               COMMAND
        CREATED             STATUS              PORTS               NAMES
        334cfa0bba29        ubuntu:latest       "/bin/sh -c 'while t   22 
minutes
        ago      Up 22 minutes                           lonely_babbage

        [root@gyarados ~]# pcp
        Performance Co-Pilot configuration on gyarados:

           platform: Linux gyarados 4.0.5-300.fc22.x86_64 #1 SMP Mon Jun 8
        16:15:26 UTC
        2015 x86_64
           hardware: 4 cpus, 3 disks, 1 node, 12011MB RAM
           timezone: CDT+5
           services: pmcd
               pmcd: Version 3.10.6-1, 7 agents
               pmda: root pmcd proc xfs linux mmv jbd2

        [root@gyarados ~]# pminfo -t --fetch --container lonely_babbage
        cgroup.memory.stat.cache proc.psinfo.rss pmcd.hostname

        cgroup.memory.stat.cache [Number of bytes of page cache memory]
        Error: Container not found

        proc.psinfo.rss [resident set size (i.e. physical memory) of the 
process]
        Error: Container not found

        pmcd.hostname [local hostname]
              value "gyarados"


    Hi Paul, good to see you've landed safely :)

    I'm not seeing these "container not found" errors - I'm running latest
    pcp-3.10.6 on f22, with two containers running named pcp-collector and
    pcp-pmlogger (see below). Do you still get the same error if you use the
    container ID rather than the name? i.e.

    # pminfo -t --fetch --container 334cfa0bba29 \
       cgroup.memory.stat.cache proc.psinfo.rss pmcd.hostname

    Here's my output for the two containers mentioned above :

    [mgoodwin@goblin ~]$ docker ps
    CONTAINER ID        IMAGE                  COMMAND             CREATED
    STATUS              PORTS               NAMES
    d0454844f478        pcp-pmlogger:latest    "pmlogger start"    45 hours ago
       Up 45 hours                             pcp-pmlogger
    ad618da9db3e        pcp-collector:latest   "pmcd start"        45 hours ago
       Up 45 hours                             pcp-collector
    [mgoodwin@goblin ~]$ pminfo -t --fetch --container pcp-pmlogger
    cgroup.memory.stat.cache proc.psinfo.rss pmcd.hostname

    cgroup.memory.stat.cache [Number of bytes of page cache memory]
    No value(s) available!

    proc.psinfo.rss [resident set size (i.e. physical memory) of the process]
         inst [14319 or "014319 /usr/libexec/pcp/bin/pmpause"] value 2112
         inst [22366 or "022366 /usr/libexec/pcp/bin/pmlogger -P -r -T24h10m -c
    config.default -m pmlogger_check 20150706.00.25"] value 4008

    pmcd.hostname [local hostname]
         value "goblin"
    [mgoodwin@goblin ~]$ pminfo -t --fetch --container pcp-collector
    cgroup.memory.stat.cache proc.psinfo.rss pmcd.hostname

    cgroup.memory.stat.cache [Number of bytes of page cache memory]
    No value(s) available!

    proc.psinfo.rss [resident set size (i.e. physical memory) of the process]
         inst [14122 or "014122 /usr/libexec/pcp/bin/pmpause"] value 2096
         inst [14190 or "014190 /usr/libexec/pcp/bin/pmcd -A"] value 3760
         inst [14192 or "014192 /var/lib/pcp/pmdas/root/pmdaroot"] value 3540
         inst [14193 or "014193 /var/lib/pcp/pmdas/proc/pmdaproc -d 3"] value 
4028
         inst [14194 or "014194 /var/lib/pcp/pmdas/xfs/pmdaxfs -d 11"] value 
3336
         inst [14195 or "014195 /var/lib/pcp/pmdas/linux/pmdalinux"] value 3720

    pmcd.hostname [local hostname]
         value "goblin"





--
--pvn

<Prev in Thread] Current Thread [Next in Thread>