From kenj@internode.on.net Tue Mar 1 00:11:04 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
by oss.sgi.com (Postfix) with ESMTP id DB0C77CA0
for ; Tue, 1 Mar 2016 00:11:03 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
by relay3.corp.sgi.com (Postfix) with ESMTP id 4F11CAC001
for ; Mon, 29 Feb 2016 22:11:00 -0800 (PST)
X-ASG-Debug-ID: 1456812656-04cb6c58319d370001-S8gJnT
Received: from ipmail06.adl6.internode.on.net (ipmail06.adl6.internode.on.net [150.101.137.145]) by cuda.sgi.com with ESMTP id AW4IxcdrhyG7YAzG for ; Mon, 29 Feb 2016 22:10:56 -0800 (PST)
X-Barracuda-Envelope-From: kenj@internode.on.net
X-Barracuda-Apparent-Source-IP: 150.101.137.145
Received: from ppp118-209-45-132.lns20.mel4.internode.on.net (HELO [192.168.1.100]) ([118.209.45.132])
by ipmail06.adl6.internode.on.net with ESMTP; 01 Mar 2016 16:40:00 +1030
To: pcp@oss.sgi.com
Subject: pcp updates: Mac OS X changes
From: Ken McDonell
X-ASG-Orig-Subj: pcp updates: Mac OS X changes
Message-ID: <56D53207.5060005@internode.on.net>
Date: Tue, 1 Mar 2016 17:09:11 +1100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
Thunderbird/38.5.1
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Barracuda-Connect: ipmail06.adl6.internode.on.net[150.101.137.145]
X-Barracuda-Start-Time: 1456812656
X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27460
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
Changes committed to git://git.pcp.io/kenj/pcp master
Ken McDonell (10):
qa/archives/GNUmakefile.install: use QA version of SUBDIRS rule
qa/024: tweaks to fix Mac OS X failures
Mac OS X build: fix a number of problems
qa/admin/myconfigure: fix for Mac OS X
Makepkgs: fix a comment that was no longer correct
qa/src/torture_indom.c: guard against bad malloc() call
src/libpcp/src/logutil.c: pander to gcc on Mac OS X
src/libpcp/src/lock.c: fix initialization error in roll-your-own TPD
qa/079: you'd think od -c was pretty standard
qa/232: skip some metrics that are not available on Mac OS X
Makepkgs | 2
build/mac/cmdline-install | 2
build/mac/install-pcp | 56 +++++++++++++++++++--------
build/mac/installer-resources/postupgrade | 36 ++++++++---------
build/mac/uninstall-pcp.in | 2
qa/024 | 19 ++++++---
qa/024.out | 3 -
qa/079 | 6 ++
qa/079.out | 10 ++--
qa/232 | 2
qa/admin/myconfigure | 61 ++++++++++++++++++++++--------
qa/archives/GNUmakefile.install | 2
qa/src/torture_indom.c | 13 ++++--
src/libpcp/src/lock.c | 1
src/libpcp/src/logutil.c | 12 ++---
15 files changed, 147 insertions(+), 80 deletions(-)
Details ...
commit 6f604c52c59f62c87884309e8f5f941cb1b71239
Author: Ken McDonell
Date: Tue Mar 1 17:06:00 2016 +1100
qa/232: skip some metrics that are not available on Mac OS X
commit c84abeced8781758b2a878807892c21810716876
Author: Ken McDonell
Date: Tue Mar 1 17:04:35 2016 +1100
qa/079: you'd think od -c was pretty standard
But not for the "we're smarter than everyone else" pixies at Apple,
so of course on Mac OS X it has gratuitous white space differences
in the output.
commit b1582f26d9b7e0a770f6dcc9f9c9a2fc409a4dd5
Author: Ken McDonell
Date: Tue Mar 1 17:01:58 2016 +1100
src/libpcp/src/lock.c: fix initialization error in roll-your-own TPD
If the compiler does not support thread-private-data, then we do
our own implementation. This is not used on most platforms.
But the code contained a lurking intialization error and on Mac OS X
it blew up. Fix is simple, and memset() is your friend.
commit 7d425b7dc01479c8e9bc07ae9e8ea0d649079a6f
Author: Ken McDonell
Date: Tue Mar 1 17:01:19 2016 +1100
src/libpcp/src/logutil.c: pander to gcc on Mac OS X
To make bogus compiler warnings go away.
commit 3f9a8e1dc59bc9fb669702fdca6fbff5b294c151
Author: Ken McDonell
Date: Tue Mar 1 16:54:38 2016 +1100
qa/src/torture_indom.c: guard against bad malloc() call
Found with malloc failure on Mac OS X.
commit 35abdf4e1a54d5226a69f0d2aab0f56a7ee5af64
Author: Ken McDonell
Date: Tue Mar 1 16:53:47 2016 +1100
Makepkgs: fix a comment that was no longer correct
commit 3e378e982f376f3385f912a6a4e88c916b4ac814
Author: Ken McDonell
Date: Tue Mar 1 16:51:29 2016 +1100
qa/admin/myconfigure: fix for Mac OS X
Makepkgs myconfigure were using an entirely different set of configopts
(the Makepkgs ones are correct, so fix myconfigure).
commit 84c15236afb3a3de88b922c07bef1119c3d716d8
Author: Ken McDonell
Date: Tue Mar 1 16:47:46 2016 +1100
Mac OS X build: fix a number of problems
This is for the old 10.8.0 version of Mac OS X.
The installation scripts were not cleaning up the previous install
... in particular binaries in /usr/bin were left behind, meaning the
newer ones in /usr/local/bin were never found for most common $PATH
setups.
Also there was a lot of useless verbage in the installer logs.
commit d7bb226d82ab979e9046b5977eb5d939e8cb36ee
Author: Ken McDonell
Date: Tue Mar 1 09:04:56 2016 +1100
qa/024: tweaks to fix Mac OS X failures
commit 1b88a442f073f5012036d4afec70970926b446de
Author: Ken McDonell
Date: Tue Mar 1 09:04:03 2016 +1100
qa/archives/GNUmakefile.install: use QA version of SUBDIRS rule
Avoids === multi === output line from check.
From myllynen@redhat.com Tue Mar 1 07:11:10 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
by oss.sgi.com (Postfix) with ESMTP id 335F67CB3
for ; Tue, 1 Mar 2016 07:11:10 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
by relay2.corp.sgi.com (Postfix) with ESMTP id 05561304032
for ; Tue, 1 Mar 2016 05:11:06 -0800 (PST)
X-ASG-Debug-ID: 1456837862-04cbb00b6ff1120001-S8gJnT
Received: from mail-wm0-f50.google.com (mail-wm0-f50.google.com [74.125.82.50]) by cuda.sgi.com with ESMTP id to6CA7Q5AArJBx24 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Tue, 01 Mar 2016 05:11:03 -0800 (PST)
X-Barracuda-Envelope-From: myllynen@redhat.com
X-Barracuda-Apparent-Source-IP: 74.125.82.50
Received: by mail-wm0-f50.google.com with SMTP id p65so32785699wmp.0
for ; Tue, 01 Mar 2016 05:11:03 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20130820;
h=x-gm-message-state:reply-to:subject:references:to:cc:from
:organization:message-id:date:user-agent:mime-version:in-reply-to
:content-transfer-encoding;
bh=/Vc6Rz4spiQ+GhyGU2SSBRBPFz3W5+bTv0Hjtzfdpas=;
b=I5AeGMVqoc6I6MZ3QwibzQbBV6QEozLzRtGH4Sf7qy/M91VjyHWxQe2Icx59OXW4Aw
6rCuMXsJNf/1snArvm+0j1p766uiKbZk8F2VgeSERNO+gsJB9Zfo97ubz0Nw/0IFvOtj
+5qXRcO5pb2e+XmiY/vw5htY/gJOY2LTGZyRSaeTM98IysTFpOaInl72Te3B3HZGnPqP
iIftp4jm4ypcnASmB/iPiDcHMo/j9LVLWd+Tkyz/CG2/Pb8gkp3rviRDt3LdJ4lh/ysR
n/YCjouVYMf5aieXzYVfe3TAiQGFNuYIRzvdCXBZVq3znyJtFmzVRYVfmzXM/1rxu1Ra
8ftA==
X-Gm-Message-State: AD7BkJKmeEUn3rGvsk0FqfrMfyrc8vKoj9MCashbDuj7wBvta65kVs4MszgwkskGy6FLwX0I
X-Received: by 10.28.215.16 with SMTP id o16mr3576156wmg.57.1456837856322;
Tue, 01 Mar 2016 05:10:56 -0800 (PST)
Received: from [192.168.1.100] (176-93-226-73.bb.dnainternet.fi. [176.93.226.73])
by smtp.gmail.com with ESMTPSA id k125sm16359380wmb.14.2016.03.01.05.10.55
(version=TLSv1/SSLv3 cipher=OTHER);
Tue, 01 Mar 2016 05:10:55 -0800 (PST)
Reply-To: Marko Myllynen
Subject: Re: [pcp] pmda leftover processes after pmda removal
References: <56D49B88.3050205@redhat.com>
<631903287.26016737.1456804286017.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: [pcp] pmda leftover processes after pmda removal
To: Nathan Scott
Cc: pcp developers
From: Marko Myllynen
Organization: Red Hat
Message-ID: <56D594DD.2070308@redhat.com>
Date: Tue, 1 Mar 2016 15:10:53 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
Thunderbird/38.6.0
MIME-Version: 1.0
In-Reply-To: <631903287.26016737.1456804286017.JavaMail.zimbra@redhat.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Barracuda-Connect: mail-wm0-f50.google.com[74.125.82.50]
X-Barracuda-Start-Time: 1456837863
X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256
X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.01
X-Barracuda-Spam-Status: No, SCORE=0.01 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_SA_TO_FROM_DOMAIN_MATCH
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27467
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header
0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient
Domain
Hi,
On 2016-03-01 05:51, Nathan Scott wrote:
> ----- Original Message -----
>> [...] I'd expect them both getting killed. Is this a feature or a
>> bug or something that should be handled in the Remove script?
>
> Not seeing this here - can you share the Install & Remove scripts
> you're using there too? Thanks.
>
> After the Remove script is run, there should definitely be no trace
> of the PMDA or any children anymore (has pmcd.conf not been updated
> somehow? maybe check the "iam=xxx" setting in Remove script?).
Install/Remove are just like everywhere else, PMDA installed as
collector, tried enabling/disabling daemon_opt/forced_restart to no avail:
Install:
. $PCP_DIR/etc/pcp.env
. $PCP_SHARE_DIR/lib/pmdaproc.sh
iam=test
perl_opt=true
daemon_opt=false
forced_restart=true
pmdaSetup
pmdaInstall
exit 0
Remove:
. $PCP_DIR/etc/pcp.env
. $PCP_SHARE_DIR/lib/pmdaproc.sh
iam=test
pmdaSetup
pmdaRemove
exit 0
pmcd.conf hasn't been updated, this happens also on a freshly booted
machine when doing the test more or less as the first thing. Log for the
test PMDA show it stopped logging when it was removed
...
[Tue Mar 1 14:55:53] pmdatest(11628) Info: Incoming data:DATA:
[Tue Mar 1 14:55:58] pmdatest(11628) Info: Incoming data:DATA:
Log finished Tue Mar 1 14:55:59 2016
But in pmcd.log I see:
pmcd RESTARTED at Tue Mar 1 14:55:59 2016
...
[Tue Mar 1 14:55:59] pmcd(11617) Info: CleanupAgent ...
Cleanup "test" agent (dom 497): unconfigured, exit(0)
...
[Tue Mar 1 14:56:00] pmcd(11617) Info: PMNS file "DEFAULT" is unchanged
...
If I stop and start pmcd with systemctl stop/start pmcd then the test
PMDA is gone as expected. So I wonder what is this cleanup agent? Should
the domain be configured somewhere else than in Install, too? Or perhaps
the QA tests I've run on this system have left some scars behind?
Thanks,
--
Marko Myllynen
From myllynen@redhat.com Tue Mar 1 10:33:41 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
by oss.sgi.com (Postfix) with ESMTP id 1ED727CB4
for ; Tue, 1 Mar 2016 10:33:41 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
by relay2.corp.sgi.com (Postfix) with ESMTP id E2267304032
for ; Tue, 1 Mar 2016 08:33:40 -0800 (PST)
X-ASG-Debug-ID: 1456850015-04cbb00b70100320001-S8gJnT
Received: from mail-wm0-f54.google.com (mail-wm0-f54.google.com [74.125.82.54]) by cuda.sgi.com with ESMTP id GkmwLmMMhWmDtpQC (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Tue, 01 Mar 2016 08:33:36 -0800 (PST)
X-Barracuda-Envelope-From: myllynen@redhat.com
X-Barracuda-Apparent-Source-IP: 74.125.82.54
Received: by mail-wm0-f54.google.com with SMTP id l68so41299942wml.1
for ; Tue, 01 Mar 2016 08:33:35 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20130820;
h=x-gm-message-state:reply-to:to:from:subject:organization:message-id
:date:user-agent:mime-version:content-transfer-encoding;
bh=DJe3EF923hnIOVSkyigZooQzJSSZ/RU/FLU4dlNEn00=;
b=PBqCAtCajIw51mTwZysKLcxJzbecy7jd41Y0NUySqFfRgTE+hlm+nammkvNHYVaJNe
7TJ6vrkgw6EazMg8wop0MhhbHeer7tfzXRjmEWDTNG4gXG4JliNafQfa9/oUBnEk8VRL
U6R18OsmobVKQx11gUa32KrWg7ZUsd2WPxne24LnuzLqa6edCI9XX27P3lzrew+UYAGM
f4slDQu4rzlqpt0Ra8xKxasRE2rm7Xz4pd9AAvoqkqoYbMa9qh8xtks0tvDL4Ahos6JD
ZD/1xDq+ok3l4NCTiMiWDDCtiJ37P5HVBRV4fdBJ3T9/WxpDsQU4OiUupTxVHmaVLL4C
kg9g==
X-Gm-Message-State: AD7BkJJZhzTKI8R1sPAkFNh//u9jyBbTN6c2c8CISwPkMjm+mWIxW1qg8jzoguKI8GWkmKq0
X-Received: by 10.194.2.41 with SMTP id 9mr23741469wjr.10.1456850014914;
Tue, 01 Mar 2016 08:33:34 -0800 (PST)
Received: from [192.168.1.100] (176-93-226-73.bb.dnainternet.fi. [176.93.226.73])
by smtp.gmail.com with ESMTPSA id v68sm15509066wmv.10.2016.03.01.08.33.33
for
(version=TLSv1/SSLv3 cipher=OTHER);
Tue, 01 Mar 2016 08:33:34 -0800 (PST)
Reply-To: Marko Myllynen
To: pcp developers
From: Marko Myllynen
Subject: Fix Perl PMDA Socket Input
Organization: Red Hat
X-ASG-Orig-Subj: Fix Perl PMDA Socket Input
Message-ID: <56D5C45C.2050600@redhat.com>
Date: Tue, 1 Mar 2016 18:33:32 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
Thunderbird/38.6.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Barracuda-Connect: mail-wm0-f54.google.com[74.125.82.54]
X-Barracuda-Start-Time: 1456850015
X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256
X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27470
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
Hi,
While reviewing the Perl PMDA pipe input code (basically it doesn't
work, I'm randomly getting empty lines or seeing lines missed which
never happens with the external command and tee(1)), I noticed the
following socket input related bug. This makes me wonder how well tested
the pipe/socket input mechanisms are, are they worth trying or is it
best to stick with the more widely used methods? (I see, for example,
that the pipe input mechanism has only three in-tree users and those
PMDAs haven't been touched for years.)
---
src/perl/PMDA/local.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/perl/PMDA/local.c b/src/perl/PMDA/local.c
index cc9956b..6d6f8c9 100644
--- a/src/perl/PMDA/local.c
+++ b/src/perl/PMDA/local.c
@@ -343,7 +343,7 @@ local_connection(files_t *file)
{
if (file->type == FILE_TAIL)
local_log_rotated(file);
- else if (file->type == FILE_TAIL)
+ else if (file->type == FILE_SOCK)
local_reconnector(file);
}
Thanks,
--
Marko Myllynen
From wwwrun@oss.sgi.com Tue Mar 1 13:37:01 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE,NO_RELAYS
autolearn=ham version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: by oss.sgi.com (Postfix, from userid 30)
id D35AA7CBD; Tue, 1 Mar 2016 13:37:01 -0600 (CST)
From: bugzilla-daemon@oss.sgi.com
To: pcp@oss.sgi.com
Subject: [Bug 1137] New: activate more compiler warnings
Date: Tue, 01 Mar 2016 19:37:01 +0000
X-Bugzilla-Reason: CC AssignedTo
X-Bugzilla-Type: new
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Classification: Unclassified
X-Bugzilla-Product: pcp
X-Bugzilla-Component: pcp
X-Bugzilla-Keywords:
X-Bugzilla-Severity: major
X-Bugzilla-Who: fche@redhat.com
X-Bugzilla-Status: NEW
X-Bugzilla-Priority: P5
X-Bugzilla-Assigned-To: pcp@oss.sgi.com
X-Bugzilla-Target-Milestone: ---
X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform
op_sys bug_status bug_severity priority component assigned_to reporter cc
classification
Message-ID:
Content-Type: multipart/alternative; boundary="1456861021.4d8eCED0.28125"; charset="us-ascii"
X-Bugzilla-URL: http://oss.sgi.com/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
--1456861021.4d8eCED0.28125
Date: Tue, 1 Mar 2016 13:37:01 -0600
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
http://oss.sgi.com/bugzilla/show_bug.cgi?id=1137
Bug ID: 1137
Summary: activate more compiler warnings
Product: pcp
Version: unspecified
Hardware: All
OS: Linux
Status: NEW
Severity: major
Priority: P5
Component: pcp
Assignee: pcp@oss.sgi.com
Reporter: fche@redhat.com
CC: pcp@oss.sgi.com
Classification: Unclassified
With gcc6, a variety of new high-quality warnings are available,
some of which have helped find genuine bugs. (For example, it
finds the file->type == FILE_TAIL duplication in perl/PMDA/local.c).
We should get the autoconf.* machinery to activate
-Wextra
-Wduplicated-cond
and probably others, if $CC supports them.
--
You are receiving this mail because:
You are on the CC list for the bug.
You are the assignee for the bug.
--1456861021.4d8eCED0.28125
Date: Tue, 1 Mar 2016 13:37:01 -0600
MIME-Version: 1.0
Content-Type: text/html; charset="UTF-8"
| Bug ID |
1137
|
| Summary |
activate more compiler warnings
|
| Product |
pcp
|
| Version |
unspecified
|
| Hardware |
All
|
| OS |
Linux
|
| Status |
NEW
|
| Severity |
major
|
| Priority |
P5
|
| Component |
pcp
|
| Assignee |
pcp@oss.sgi.com
|
| Reporter |
fche@redhat.com
|
| CC |
pcp@oss.sgi.com
|
| Classification |
Unclassified
|
With gcc6, a variety of new high-quality warnings are available,
some of which have helped find genuine bugs. (For example, it
finds the file->type == FILE_TAIL duplication in perl/PMDA/local.c).
We should get the autoconf.* machinery to activate
-Wextra
-Wduplicated-cond
and probably others, if $CC supports them.
You are receiving this mail because:
- You are on the CC list for the bug.
- You are the assignee for the bug.
--1456861021.4d8eCED0.28125--
From nscott@redhat.com Tue Mar 1 15:23:13 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
by oss.sgi.com (Postfix) with ESMTP id AAF4F7CBB
for ; Tue, 1 Mar 2016 15:23:13 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
by relay1.corp.sgi.com (Postfix) with ESMTP id 5E8EC8F8035
for ; Tue, 1 Mar 2016 13:23:13 -0800 (PST)
X-ASG-Debug-ID: 1456867387-04bdf002ebfc3f0001-S8gJnT
Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id srKqR3lYoWEDKQ3x (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 01 Mar 2016 13:23:08 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.25
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id u21LN7bE021385;
Tue, 1 Mar 2016 16:23:07 -0500
Date: Tue, 1 Mar 2016 16:23:06 -0500 (EST)
From: Nathan Scott
Reply-To: Nathan Scott
To: Marko Myllynen
Cc: pcp developers
Message-ID: <896217520.26492556.1456867386925.JavaMail.zimbra@redhat.com>
In-Reply-To: <56D5C45C.2050600@redhat.com>
References: <56D5C45C.2050600@redhat.com>
Subject: Re: [pcp] Fix Perl PMDA Socket Input
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] Fix Perl PMDA Socket Input
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.64.48.155]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF44 (Linux)/8.0.6_GA_5922)
Thread-Topic: Fix Perl PMDA Socket Input
Thread-Index: RY4bgLl7eRAknF1SId6qNE8QuZQa0A==
X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25]
X-Barracuda-Start-Time: 1456867388
X-Barracuda-Encrypted: DHE-RSA-AES256-SHA
X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.03
X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27481
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)...
0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header
0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient
Domain
----- Original Message -----
> Hi,
>
> While reviewing the Perl PMDA pipe input code (basically it doesn't
> work, I'm randomly getting empty lines or seeing lines missed which
> never happens with the external command and tee(1)), I noticed the
> following socket input related bug. This makes me wonder how well tested
> the pipe/socket input mechanisms are, are they worth trying or is it
> best to stick with the more widely used methods?
Best would be to fix the bug(s) as you've done - also good would be to
have some modern PMDAs using the code to exercise it, and/or some more
regression tests in the area. :)
Thanks Marko!
cheers.
--
Nathan
From myllynen@redhat.com Tue Mar 1 15:57:21 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
by oss.sgi.com (Postfix) with ESMTP id E29B17CBB
for ; Tue, 1 Mar 2016 15:57:21 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
by relay2.corp.sgi.com (Postfix) with ESMTP id 96518304032
for ; Tue, 1 Mar 2016 13:57:18 -0800 (PST)
X-ASG-Debug-ID: 1456869434-04bdf002ecfed60001-S8gJnT
Received: from mail-wm0-f52.google.com (mail-wm0-f52.google.com [74.125.82.52]) by cuda.sgi.com with ESMTP id zPKTSHzUWNgUoemA (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Tue, 01 Mar 2016 13:57:15 -0800 (PST)
X-Barracuda-Envelope-From: myllynen@redhat.com
X-Barracuda-Apparent-Source-IP: 74.125.82.52
Received: by mail-wm0-f52.google.com with SMTP id l68so55070449wml.0
for ; Tue, 01 Mar 2016 13:57:15 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20130820;
h=x-gm-message-state:reply-to:subject:references:to:cc:from
:organization:message-id:date:user-agent:mime-version:in-reply-to
:content-transfer-encoding;
bh=HJOQhmR6s1XqfGTER4vwLxc5kNCbgnTUhOXGqlI6A0k=;
b=IycQ5jFygtd4WwTsLS0vggSU6qpmZc/ga5REXy9tdhLUAVv7629NVkttRZz6BhteyQ
ikXqWlDLn4975ZUgaPBSOQGkQcLdK3RI6yL8ivamc3RbU6NOwAWYyutPOnJiel49xwDD
uhJNDcrvWyUmkWVmh9t6Em3WG9nwXwWI+QXlU68NWjlPyFjMul4qerjfekqYa7tpnwMV
2HcX1EelQLjeOlLHfiS/jNqt/Odsi8B9xjCGMMJVh0nGYFWmNmoWAiTBzEqtnioLagVw
fwNuHvpDDTDnrvwe0Is3IbM9NPpgbdzqHxVhfpevQTHjP2prYttG5mN1D8Ldwhmo7MPH
8bIQ==
X-Gm-Message-State: AD7BkJL5PATi0YitQyc/T0SyujLaJR+tikbQozE49aObYK8mPOq2kTTGvjBhx9h5nzNnlkJX
X-Received: by 10.194.94.106 with SMTP id db10mr22495195wjb.175.1456869434590;
Tue, 01 Mar 2016 13:57:14 -0800 (PST)
Received: from [192.168.1.100] (176-93-226-73.bb.dnainternet.fi. [176.93.226.73])
by smtp.gmail.com with ESMTPSA id 192sm1109228wmw.0.2016.03.01.13.57.13
(version=TLSv1/SSLv3 cipher=OTHER);
Tue, 01 Mar 2016 13:57:13 -0800 (PST)
Reply-To: Marko Myllynen
Subject: Re: [pcp] Fix Perl PMDA Socket Input
References: <56D5C45C.2050600@redhat.com>
<896217520.26492556.1456867386925.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: [pcp] Fix Perl PMDA Socket Input
To: Nathan Scott
Cc: pcp developers
From: Marko Myllynen
Organization: Red Hat
Message-ID: <56D61037.8060300@redhat.com>
Date: Tue, 1 Mar 2016 23:57:11 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
Thunderbird/38.6.0
MIME-Version: 1.0
In-Reply-To: <896217520.26492556.1456867386925.JavaMail.zimbra@redhat.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Barracuda-Connect: mail-wm0-f52.google.com[74.125.82.52]
X-Barracuda-Start-Time: 1456869435
X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256
X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.01
X-Barracuda-Spam-Status: No, SCORE=0.01 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_SA_TO_FROM_DOMAIN_MATCH
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27482
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header
0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient
Domain
Hi,
On 2016-03-01 23:23, Nathan Scott wrote:
> ----- Original Message -----
>>
>> While reviewing the Perl PMDA pipe input code (basically it doesn't
>> work, I'm randomly getting empty lines or seeing lines missed which
>> never happens with the external command and tee(1)), I noticed the
>> following socket input related bug. This makes me wonder how well tested
>> the pipe/socket input mechanisms are, are they worth trying or is it
>> best to stick with the more widely used methods?
>
> Best would be to fix the bug(s) as you've done - also good would be to
> have some modern PMDAs using the code to exercise it, and/or some more
> regression tests in the area. :)
If still using the earlier sent pmdatest.pl but with the below script
to produce "CSV" data, I'm starting to see discrepancies in the
received data immediately - looks like the key factor is the length of
the input lines:
#!/bin/sh
echo "# Timestamp;ID;Key;Value;Type"
while true; do
ts="A very long time timestamp here for the sake of testing"
echo "# BOD"
for i in $(seq 1 99); do
echo "$ts;$i;Date;$ts$ts$ts;String"
done
echo "# EOD"
sleep 5
done
And this is what the PMDA prints (obviously works ok on cmd line):
...
Info: Incoming data:A very long time timestamp here for the sake of testing;88;Date;A very long time timestamp here for the sake of testingA very long time timestamp here for the sake of testingA very long time timestamp here for the sake of testing;String:
Info: Incoming data:the sake of testingA very long time timestamp here for the sake of testingA very long time timestamp here for the sake of testing;String:
Info: Incoming data:A very long time timestamp here for the sake of testing;90;Date;A very long time timestamp here for the sake of testingA very long time timestamp here for the sake of testingA very long time timestamp here for the sake of testing;String:
...
Thanks,
--
Marko Myllynen
From kenj@kenj.com.au Tue Mar 1 22:47:30 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: **
X-Spam-Status: No, score=2.4 required=5.0 tests=DATE_IN_PAST_03_06
autolearn=no version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
by oss.sgi.com (Postfix) with ESMTP id BA1D77CBB
for ; Tue, 1 Mar 2016 22:47:30 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
by relay3.corp.sgi.com (Postfix) with ESMTP id 3CE64AC003
for ; Tue, 1 Mar 2016 20:47:30 -0800 (PST)
X-ASG-Debug-ID: 1456894041-04cbb00b71129490001-S8gJnT
Received: from ipmail06.adl2.internode.on.net (ipmail06.adl2.internode.on.net [150.101.137.129]) by cuda.sgi.com with ESMTP id 6NAB91kFUQEIe8vr for ; Tue, 01 Mar 2016 20:47:23 -0800 (PST)
X-Barracuda-Envelope-From: kenj@kenj.com.au
X-Barracuda-Apparent-Source-IP: 150.101.137.129
Received: from ppp118-209-12-53.lns20.mel4.internode.on.net (HELO kenj) ([118.209.12.53])
by ipmail06.adl2.internode.on.net with ESMTP; 02 Mar 2016 15:17:20 +1030
Received: by kenj (Postfix, from userid 1000)
id A0194A0BE2; Wed, 2 Mar 2016 12:40:47 +1100 (AEDT)
To: pcp@oss.sgi.com
Subject: pcp updates: misc
Message-Id: <20160302014047.A0194A0BE2@kenj>
X-ASG-Orig-Subj: pcp updates: misc
Date: Wed, 2 Mar 2016 12:40:47 +1100 (AEDT)
From: kenj@kenj.com.au (Ken McDonell)
X-Barracuda-Connect: ipmail06.adl2.internode.on.net[150.101.137.129]
X-Barracuda-Start-Time: 1456894041
X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 1.09
X-Barracuda-Spam-Status: No, SCORE=1.09 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=DATE_IN_PAST_03_06, DATE_IN_PAST_03_06_2
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27494
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
0.01 DATE_IN_PAST_03_06 Date: is 3 to 6 hours before Received: date
1.08 DATE_IN_PAST_03_06_2 DATE_IN_PAST_03_06_2
Changes committed to git://git.pcp.io/kenj/pcp master
Ken McDonell (3):
qa/324: notrun guard if pmda not installed and cleanup shm segment
assorted qa tests: add notrun check for PMDA dir before cd'ing there
qa/992: fix broken PMID filter
qa/1017 | 1 +
qa/1031 | 1 +
qa/137 | 2 ++
qa/200 | 28 ++++++++++++++--------------
qa/241 | 1 +
qa/255 | 2 ++
qa/273 | 2 ++
qa/308 | 2 ++
qa/324 | 3 +++
qa/347 | 2 ++
qa/455 | 7 ++++---
qa/560 | 2 ++
qa/572 | 4 ++++
qa/578 | 2 ++
qa/587 | 2 ++
qa/603 | 2 ++
qa/628 | 2 ++
qa/642 | 5 ++++-
qa/643 | 2 ++
qa/657 | 1 +
qa/716 | 2 ++
qa/717 | 3 +++
qa/718 | 2 ++
qa/721 | 2 ++
qa/725 | 1 +
qa/778 | 2 ++
qa/788 | 4 ++--
qa/798 | 2 ++
qa/805 | 3 ++-
qa/825 | 2 ++
qa/835 | 2 ++
qa/845 | 3 +++
qa/960 | 2 ++
qa/964 | 2 ++
qa/976 | 3 ++-
qa/981 | 2 ++
qa/992 | 6 +++---
qa/992.out | 12 ++++++------
38 files changed, 97 insertions(+), 31 deletions(-)
Details ...
commit e382507571c5dc9077a09add0bd5c75cdf587990
Author: Ken McDonell
Date: Wed Mar 2 10:54:23 2016 +1100
qa/992: fix broken PMID filter
commit 420ac93c1d2a136254b1dea5bf9564fd42450efc
Author: Ken McDonell
Date: Wed Mar 2 10:08:19 2016 +1100
assorted qa tests: add notrun check for PMDA dir before cd'ing there
Missing guards were producing some very strange results on Mac OS X
where some PMDAs are not built and installed.
commit 5e2cb168016bce45cfcafbb6d740840390552685
Author: Ken McDonell
Date: Wed Mar 2 08:03:20 2016 +1100
qa/324: notrun guard if pmda not installed and cleanup shm segment
From charles.cui1984@gmail.com Wed Mar 2 15:46:57 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.3 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,
FREEMAIL_FROM,HTML_MESSAGE,T_DKIM_INVALID autolearn=no version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
by oss.sgi.com (Postfix) with ESMTP id 54ADF7CB3
for ; Wed, 2 Mar 2016 15:46:57 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
by relay2.corp.sgi.com (Postfix) with ESMTP id 27414304053
for ; Wed, 2 Mar 2016 13:46:54 -0800 (PST)
X-ASG-Debug-ID: 1456955208-04cbb00b7015e470001-S8gJnT
Received: from mail-lf0-f51.google.com (mail-lf0-f51.google.com [209.85.215.51]) by cuda.sgi.com with ESMTP id IXu80OpBzzBerzk8 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Wed, 02 Mar 2016 13:46:49 -0800 (PST)
X-Barracuda-Envelope-From: charles.cui1984@gmail.com
X-Barracuda-Apparent-Source-IP: 209.85.215.51
Received: by mail-lf0-f51.google.com with SMTP id l83so1457175lfd.3
for ; Wed, 02 Mar 2016 13:46:49 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20120113;
h=mime-version:date:message-id:subject:from:to;
bh=H+0dAP+qLWMO/0LVl9wLJpGJBSVqQZdg97ApIXg0J00=;
b=fk1VUeja71nUIauI5bTEBblO/mXjWZiYVdIQe6hoX2cvphqzw8rbF7n27RlDues1Ho
M65aGZWJYRS1tdx8BX738ffWxh0Xapm0EooBFxIHpfLB7A/1remlBEII3oqHDm7qLI/r
QjQQJNBJVjMNHF8UujfiLwdfp/JtTw5OADoUzRXJwsEq3m1ngmbIs6lhvEF1e+SB7l2j
wh5gqY06gvLBgdd/LKsVQ8MfXm1eG4SYR5bsvtEzU+LACTvve7VSOH+iLvTpoz0qZd4p
8YLuCVSbawaih6HEL/9GWk9yICGIG0RMFQ6JzLyO6y3eQAPTiwi4y2R/DILvYtxyHh50
0Xsg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20130820;
h=x-gm-message-state:mime-version:date:message-id:subject:from:to;
bh=H+0dAP+qLWMO/0LVl9wLJpGJBSVqQZdg97ApIXg0J00=;
b=aSJLAg5GhZd8BOkcfBmjPq8YGLmGPmoM2b6/JodUqni/GT4B8I++iHmj5pU28n//ZG
1KlNoATV/5roBd6NOXdxTHxzz+SEA3gpkoVhlTzBpoyqRMQq7BpOM/qrlMskPIdRRcbb
IsQzi2kwBU7Og4AoeA6ZAF0jxRegY/H9xZwYNICXJOdg9Kl9DUG4KpDDy6P6iwfvqiec
pWeqmUVF/ZclbLG2LRhmbhKdtImnheCJ8Q0ZQZXf94AHY5ujBcujcaBGX+BcjxvYoReO
BmO6ffkH1UJzmzxBmyOAVgsxBLyy5Jft7AcdSxEYMO1mMkQzQeCp6EZxouJ2NSJrBHvW
ODVw==
X-Gm-Message-State: AD7BkJJPuRLKsbbQZjQ6PbpOQMgeU08ZelfCGZUVbFNmy3uVj168j3tkAcv8qtMqy4ebagFFJSr+hYA/wUzdpg==
MIME-Version: 1.0
X-Received: by 10.25.170.85 with SMTP id t82mr8792966lfe.57.1456955208209;
Wed, 02 Mar 2016 13:46:48 -0800 (PST)
Received: by 10.25.86.68 with HTTP; Wed, 2 Mar 2016 13:46:48 -0800 (PST)
Date: Wed, 2 Mar 2016 13:46:48 -0800
Message-ID:
Subject: GSOC 2016 idea for PCP
From: Charles Cui
X-ASG-Orig-Subj: GSOC 2016 idea for PCP
To: pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=001a114113e8a39c0f052d17d1ae
X-Barracuda-Connect: mail-lf0-f51.google.com[209.85.215.51]
X-Barracuda-Start-Time: 1456955209
X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256
X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=DKIM_SIGNED, DKIM_VERIFIED, HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27516
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
-0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes
verification
0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature
0.00 HTML_MESSAGE BODY: HTML included in message
--001a114113e8a39c0f052d17d1ae
Content-Type: text/plain; charset=UTF-8
Hi Guys,
This is Charles Cui from Tsinghua University, China. I am pretty
interested in the ideas listed in GSOC 2016 pages for PCP, especially the
idea of "performance metric extraction using go language agents". As a PhD
student in Tsinghua, I used C for roughly 4 years. At the same time, I have
golang experience for the production code. Besides, I speak fluent English
and am available the full summer. I am not sure whether I can join in the
development of PCP? In case my project is selected by others, I am able to
pick others.
Thanks, Charles.
--001a114113e8a39c0f052d17d1ae
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Hi Guys,=C2=A0
=C2=A0 =C2=A0 This is Ch=
arles Cui from Tsinghua University, China. I am pretty interested in the id=
eas listed in GSOC 2016 pages for PCP, especially the idea of "perform=
ance metric extraction using go language agents". As a PhD student in =
Tsinghua, I used C for roughly 4 years. At the same time, I have golang exp=
erience for the production code. Besides, I speak fluent English and am ava=
ilable the full summer. I am not sure whether I can join in the development=
of PCP? In case my project is selected by others, I am able to pick others=
.
Thanks, Charles.
--001a114113e8a39c0f052d17d1ae--
From nscott@redhat.com Wed Mar 2 17:54:56 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
by oss.sgi.com (Postfix) with ESMTP id D94E77CB4
for ; Wed, 2 Mar 2016 17:54:55 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
by relay3.corp.sgi.com (Postfix) with ESMTP id 50B3DAC003
for ; Wed, 2 Mar 2016 15:54:52 -0800 (PST)
X-ASG-Debug-ID: 1456962884-04cbb00b72161bb0001-S8gJnT
Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com [209.132.183.25]) by cuda.sgi.com with ESMTP id f4VS7gWvEVV6shGd (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 02 Mar 2016 15:54:44 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.25
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id u22Nsi33023451
for ; Wed, 2 Mar 2016 18:54:44 -0500
Date: Wed, 2 Mar 2016 18:54:43 -0500 (EST)
From: Nathan Scott
Reply-To: Nathan Scott
To: pcp developers
Message-ID: <1353490033.26924432.1456962883970.JavaMail.zimbra@redhat.com>
Subject: pcp updates: kenj+marko merges, perl api fixes
MIME-Version: 1.0
X-ASG-Orig-Subj: pcp updates: kenj+marko merges, perl api fixes
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.64.48.134]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF44 (Linux)/8.0.6_GA_5922)
Thread-Topic: pcp updates: kenj+marko merges, perl api fixes
Thread-Index: kMmEQ7p4Blwum8a5XLruObsuAxIkWQ==
X-Barracuda-Connect: mx4-phx2.redhat.com[209.132.183.25]
X-Barracuda-Start-Time: 1456962884
X-Barracuda-Encrypted: DHE-RSA-AES256-SHA
X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27522
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)...
Changes committed to git://git.pcp.io/pcp.git master
Ken McDonell (13):
qa/archives/GNUmakefile.install: use QA version of SUBDIRS rule
qa/024: tweaks to fix Mac OS X failures
Mac OS X build: fix a number of problems
qa/admin/myconfigure: fix for Mac OS X
Makepkgs: fix a comment that was no longer correct
qa/src/torture_indom.c: guard against bad malloc() call
src/libpcp/src/logutil.c: pander to gcc on Mac OS X
src/libpcp/src/lock.c: fix initialization error in roll-your-own TPD
qa/079: you'd think od -c was pretty standard
qa/232: skip some metrics that are not available on Mac OS X
qa/324: notrun guard if pmda not installed and cleanup shm segment
assorted qa tests: add notrun check for PMDA dir before cd'ing there
qa/992: fix broken PMID filter
Nathan Scott (2):
build: another piece of the puzzle re gpg signing bintray repos
perl api: merge logger pmda fix from kenj commit 560c37c41a
Marko Myllynen (1):
perl api: fix input-type-socket connection logic typo
Makepkgs | 2
build/mac/cmdline-install | 2
build/mac/install-pcp | 56 +++++++++++++++++++--------
build/mac/installer-resources/postupgrade | 36 ++++++++---------
build/mac/uninstall-pcp.in | 2
qa/024 | 19 ++++++---
qa/024.out | 3 -
qa/079 | 6 ++
qa/079.out | 10 ++--
qa/1017 | 1
qa/1031 | 1
qa/137 | 2
qa/200 | 28 ++++++-------
qa/232 | 2
qa/241 | 1
qa/255 | 2
qa/273 | 2
qa/308 | 2
qa/324 | 3 +
qa/347 | 2
qa/455 | 7 +--
qa/560 | 2
qa/572 | 4 +
qa/578 | 2
qa/587 | 2
qa/603 | 2
qa/628 | 2
qa/642 | 5 +-
qa/643 | 2
qa/657 | 1
qa/716 | 2
qa/717 | 3 +
qa/718 | 2
qa/721 | 2
qa/725 | 1
qa/778 | 2
qa/788 | 4 -
qa/798 | 2
qa/805 | 3 -
qa/825 | 2
qa/835 | 2
qa/845 | 3 +
qa/960 | 2
qa/964 | 2
qa/976 | 3 -
qa/981 | 2
qa/992 | 6 +-
qa/992.out | 12 ++---
qa/admin/myconfigure | 61 ++++++++++++++++++++++--------
qa/archives/GNUmakefile.install | 2
qa/src/torture_indom.c | 13 ++++--
scripts/bintray-upload | 15 ++++---
src/libpcp/src/lock.c | 1
src/libpcp/src/logutil.c | 12 ++---
src/perl/PMDA/local.c | 8 ++-
55 files changed, 260 insertions(+), 118 deletions(-)
commit 22fdcd707d64ad35e68c057c8c66478e427c464a
Author: Nathan Scott
Date: Thu Mar 3 09:26:06 2016 +1100
perl api: merge logger pmda fix from kenj commit 560c37c41a
commit 977239e3baa5df04b9edc8ef7ec93db32e098959
Author: Marko Myllynen
Date: Thu Mar 3 09:24:22 2016 +1100
perl api: fix input-type-socket connection logic typo
commit 3566e8038d436b09141e0fc134abdecc86b7fd18
Author: Nathan Scott
Date: Thu Mar 3 09:13:27 2016 +1100
build: another piece of the puzzle re gpg signing bintray repos
commit e382507571c5dc9077a09add0bd5c75cdf587990
Author: Ken McDonell
Date: Wed Mar 2 10:54:23 2016 +1100
qa/992: fix broken PMID filter
commit 420ac93c1d2a136254b1dea5bf9564fd42450efc
Author: Ken McDonell
Date: Wed Mar 2 10:08:19 2016 +1100
assorted qa tests: add notrun check for PMDA dir before cd'ing there
Missing guards were producing some very strange results on Mac OS X
where some PMDAs are not built and installed.
commit 5e2cb168016bce45cfcafbb6d740840390552685
Author: Ken McDonell
Date: Wed Mar 2 08:03:20 2016 +1100
qa/324: notrun guard if pmda not installed and cleanup shm segment
commit 6f604c52c59f62c87884309e8f5f941cb1b71239
Author: Ken McDonell
Date: Tue Mar 1 17:06:00 2016 +1100
qa/232: skip some metrics that are not available on Mac OS X
commit c84abeced8781758b2a878807892c21810716876
Author: Ken McDonell
Date: Tue Mar 1 17:04:35 2016 +1100
qa/079: you'd think od -c was pretty standard
But not for the "we're smarter than everyone else" pixies at Apple,
so of course on Mac OS X it has gratuitous white space differences
in the output.
commit b1582f26d9b7e0a770f6dcc9f9c9a2fc409a4dd5
Author: Ken McDonell
Date: Tue Mar 1 17:01:58 2016 +1100
src/libpcp/src/lock.c: fix initialization error in roll-your-own TPD
If the compiler does not support thread-private-data, then we do
our own implementation. This is not used on most platforms.
But the code contained a lurking intialization error and on Mac OS X
it blew up. Fix is simple, and memset() is your friend.
commit 7d425b7dc01479c8e9bc07ae9e8ea0d649079a6f
Author: Ken McDonell
Date: Tue Mar 1 17:01:19 2016 +1100
src/libpcp/src/logutil.c: pander to gcc on Mac OS X
To make bogus compiler warnings go away.
commit 3f9a8e1dc59bc9fb669702fdca6fbff5b294c151
Author: Ken McDonell
Date: Tue Mar 1 16:54:38 2016 +1100
qa/src/torture_indom.c: guard against bad malloc() call
Found with malloc failure on Mac OS X.
commit 35abdf4e1a54d5226a69f0d2aab0f56a7ee5af64
Author: Ken McDonell
Date: Tue Mar 1 16:53:47 2016 +1100
Makepkgs: fix a comment that was no longer correct
commit 3e378e982f376f3385f912a6a4e88c916b4ac814
Author: Ken McDonell
Date: Tue Mar 1 16:51:29 2016 +1100
qa/admin/myconfigure: fix for Mac OS X
Makepkgs myconfigure were using an entirely different set of configopts
(the Makepkgs ones are correct, so fix myconfigure).
commit 84c15236afb3a3de88b922c07bef1119c3d716d8
Author: Ken McDonell
Date: Tue Mar 1 16:47:46 2016 +1100
Mac OS X build: fix a number of problems
This is for the old 10.8.0 version of Mac OS X.
The installation scripts were not cleaning up the previous install
... in particular binaries in /usr/bin were left behind, meaning the
newer ones in /usr/local/bin were never found for most common $PATH
setups.
Also there was a lot of useless verbage in the installer logs.
commit d7bb226d82ab979e9046b5977eb5d939e8cb36ee
Author: Ken McDonell
Date: Tue Mar 1 09:04:56 2016 +1100
qa/024: tweaks to fix Mac OS X failures
commit 1b88a442f073f5012036d4afec70970926b446de
Author: Ken McDonell
Date: Tue Mar 1 09:04:03 2016 +1100
qa/archives/GNUmakefile.install: use QA version of SUBDIRS rule
Avoids === multi === output line from check.
From pcp-announce-bounces@oss.sgi.com Wed Mar 2 18:43:02 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=RP_MATCHES_RCVD autolearn=ham
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from oss.sgi.com (localhost [IPv6:::1])
by oss.sgi.com (Postfix) with ESMTP id 7A63C7CB8;
Wed, 2 Mar 2016 18:43:02 -0600 (CST)
X-Original-To: pcp-announce@oss.sgi.com
Delivered-To: pcp-announce@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
by oss.sgi.com (Postfix) with ESMTP id 6C3827CB4
for ; Wed, 2 Mar 2016 18:43:00 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
by relay1.corp.sgi.com (Postfix) with ESMTP id 314F88F8035
for ; Wed, 2 Mar 2016 16:42:59 -0800 (PST)
X-ASG-Debug-ID: 1456965773-04cbb00b6f163610001-87ZIJf
Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by
cuda.sgi.com with ESMTP id uFIAdEdp021o6hod (version=TLSv1
cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for
; Wed, 02 Mar 2016 16:42:54 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.24
Received: from zmail20.collab.prod.int.phx2.redhat.com
(zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id u230grdN012707;
Wed, 2 Mar 2016 19:42:53 -0500
Date: Wed, 2 Mar 2016 19:42:53 -0500 (EST)
From: Nathan Scott
To: pcp-announce
Message-ID: <488657440.26930813.1456965773591.JavaMail.zimbra@redhat.com>
In-Reply-To: <876577393.26924469.1456962916667.JavaMail.zimbra@redhat.com>
MIME-Version: 1.0
X-ASG-Orig-Subj: PCP is a mentor organisation for GSoC 2016
X-Originating-IP: [10.64.48.134]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF44 (Linux)/8.0.6_GA_5922)
Thread-Topic: PCP is a mentor organisation for GSoC 2016
Thread-Index: sslerSMDcZMdkpKtjMzQObcS/Zbesg==
X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24]
X-Barracuda-Start-Time: 1456965774
X-Barracuda-Encrypted: DHE-RSA-AES256-SHA
X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No,
SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0
QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX,
THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27522
Rule breakdown below
pts rule name description
---- ----------------------
--------------------------------------------------
0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)...
Cc: Lukas Berk
Subject: [pcp-announce] PCP is a mentor organisation for GSoC 2016
X-BeenThere: pcp-announce@oss.sgi.com
X-Mailman-Version: 2.1.14
Precedence: list
Reply-To: Nathan Scott
List-Id: Performance Co-Pilot announcements
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pcp-announce-bounces@oss.sgi.com
Sender: pcp-announce-bounces@oss.sgi.com
Hi all,
The Performance Co-Pilot project has been accepted as a Google Summer
of Code mentor organization this year. Many thanks to the mentors who
have volunteered their time and expertise from all of our organization
admins (Lukas, Martha, and myself).
The PCP project list is here:
http://pcp.io/gsoc/2016/ideas.html
and we're already seeing some student interest for several ideas there
which is great!
If you know of any students who might be interested in being involved,
please forward our details on -
https://summerofcode.withgoogle.com/organizations/5748058985857024/
https://developers.google.com/open-source/gsoc/timeline
If you are an experienced PCP or Vector developer and are interested
in getting involved as a mentor, please feel free to send me a note -
there's certainly scope for additional potential projects to be added
at this early stage too.
cheers.
--
Nathan
_______________________________________________
pcp-announce mailing list
pcp-announce@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/pcp-announce
From lberk@redhat.com Wed Mar 2 20:51:07 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
by oss.sgi.com (Postfix) with ESMTP id 4A8C57CB7
for ; Wed, 2 Mar 2016 20:51:07 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
by relay1.corp.sgi.com (Postfix) with ESMTP id 1E0AD8F8040
for ; Wed, 2 Mar 2016 18:51:03 -0800 (PST)
X-ASG-Debug-ID: 1456973462-04cbb00b6f1693b0001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id 0hLNecnHgiaNAc2e (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Wed, 02 Mar 2016 18:51:02 -0800 (PST)
X-Barracuda-Envelope-From: lberk@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23])
by mx1.redhat.com (Postfix) with ESMTPS id 26BB714A80;
Thu, 3 Mar 2016 02:51:02 +0000 (UTC)
Received: from toium (vpn-55-211.rdu2.redhat.com [10.10.55.211])
by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u232p08E009307
(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO);
Wed, 2 Mar 2016 21:51:01 -0500
From: Lukas Berk
To: Charles Cui
Cc: pcp@oss.sgi.com
Subject: Re: [pcp] GSOC 2016 idea for PCP
References:
X-ASG-Orig-Subj: Re: [pcp] GSOC 2016 idea for PCP
Date: Wed, 02 Mar 2016 21:51:00 -0500
In-Reply-To:
(Charles Cui's message of "Wed, 2 Mar 2016 13:46:48 -0800")
Message-ID: <87si089svf.fsf@redhat.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1456973462
X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384
X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
Hi Charles,
Charles Cui writes:
[...]
> This is Charles Cui from Tsinghua University, China. I am pretty
> interested in the ideas listed in GSOC 2016 pages for PCP, especially
> the idea of "performance metric extraction using go language agents".
> As a PhD student in Tsinghua, I used C for roughly 4 years. At the
> same time, I have golang experience for the production code. Besides,
> I speak fluent English and am available the full summer. I am not sure
> whether I can join in the development of PCP? In case my project is
> selected by others, I am able to pick others.
Thanks for your interest in PCP! Currently, no proposed project has
been taken yet (official applications start on March 14), so no
worries there. It's great to see that you've subscribed to the
mailing list, we've also got a irc channel on freenode.net (#pcp) if
you prefer that form of communication.
Sounds like you've got some relevant programming language experience,
have you contributed to any open source projects, or Google Summer of
Code before?
A few more details about the task itself:
So, from a metrics collection standpoint, Performance Co-Pilot has two
components. The first is the pmcd (1) (Performance Metrics Collector
Daemon), which orchestrates client requests, and gathers the requested
metrics from the individual pmdas (Performance Metric Domain Agents).
Each of which define and collect a certain subset of metrics from the
system in question. The pmcd is written in C, and currently we have
pmdas (the agents), written in C/C++, perl, and python. We're looking
to extend pmdas to Golang as well, which will include writing some of
the standard pmda functions, as well as libraries, to interface between
pmdas written in Golang, to the C level apis and libraries which
currently exist.
If you're particularly keen to read more about the technical
implementation details within PCP, we've got plenty of documentation on
the website[1], specifically the Programmer's Guide[2] (Chapter Two is
specifically for writing a pmda).
Please feel free to reach out with any more specific questions you might
have, regardless if it's about the task, or the PCP community as a whole.
Cheers,
Lukas
[1] - http://pcp.io/documentation.html
[2] - http://pcp.io/books/PCP_PG/pdf/PCP-3-pcp-programmers-guide-en-US.pdf
From charles.cui1984@gmail.com Wed Mar 2 23:31:42 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.3 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,
FREEMAIL_FROM,HTML_MESSAGE,T_DKIM_INVALID autolearn=no version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
by oss.sgi.com (Postfix) with ESMTP id 9FF3A7CB7
for ; Wed, 2 Mar 2016 23:31:42 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
by relay3.corp.sgi.com (Postfix) with ESMTP id 3519CAC001
for ; Wed, 2 Mar 2016 21:31:39 -0800 (PST)
X-ASG-Debug-ID: 1456983094-04bdf002ee1601a0001-S8gJnT
Received: from mail-lb0-f171.google.com (mail-lb0-f171.google.com [209.85.217.171]) by cuda.sgi.com with ESMTP id FBK9WTeEzfY7Jfn5 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Wed, 02 Mar 2016 21:31:35 -0800 (PST)
X-Barracuda-Envelope-From: charles.cui1984@gmail.com
X-Barracuda-Apparent-Source-IP: 209.85.217.171
Received: by mail-lb0-f171.google.com with SMTP id of3so10365731lbc.1
for ; Wed, 02 Mar 2016 21:31:35 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20120113;
h=mime-version:in-reply-to:references:date:message-id:subject:from:to
:cc;
bh=5LrUgZvObc/fx19fDDIoZMkOWpLNOkgRuxFKjdKN3FE=;
b=PX3TVGWLjp+VeAgzmIeAzw7czfsLj+eURZU1V6JuPmu2P1JGVkfMtG3qNppUsfB43v
+Z03L7w61YGA7zUo/3x4ueMfM0w4/mSkgJ0o7o0ASMA8HxvavRLpOrfkX8YGUPLPuQ2+
6Yv5tFEHouSrrM8ruBWA6wkUyDUhb6HzSeAd74ps1EuzKngiVcJCTxe7n+ssln7Ilv1F
gYprIzaXeZvpbu7bs0UmD4kNadDk4ZFj+MbNMB9LVk1sdZ0MaXNhuG4mGNwQQWF6Yce+
qY3kPhS001iKdm5haszCJJXXUZFHvyBJWS2Hm+a3A7G/jk/48Z00fgkG64Vt+rHtqPIi
NGgA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20130820;
h=x-gm-message-state:mime-version:in-reply-to:references:date
:message-id:subject:from:to:cc;
bh=5LrUgZvObc/fx19fDDIoZMkOWpLNOkgRuxFKjdKN3FE=;
b=Ls0toe1NCzx8HRbI7anFq4tJbEK7P8C/SZlpO4RoiJ9Yp6ZTOqjvuaZXWfPn9wzpzG
b2fzAhVhcPepLp8l7Jt+l4ljmQ1OrMO8bEiiQpwv8VVSHdkvNE+pFHJGxwcv340CbGOV
F0RV4enyqjCE7Hz/rxqcJXX3vqeERDv0pfhGmiVCEwYQYP3I50PJ0seeJ+8PIafYgQUM
eKyZGRpRibujVK1kshpCc4aC09+705rSl/fPhJ1xNQHZ1oXP9hMnWFZ0g/J9Lt/aJBRW
jIOkDG3/Rj8fiObkf/xmSac9wZ9i4THr0jchvHaUs3maKI0EPUfGiMj+5ErmR0oC1XXJ
m5ig==
X-Gm-Message-State: AD7BkJKuybDI43OehGUAnYubHRYToDJPcDJW7ZaZ9KzGNmv8SpgEOQC+VvkFDG8nNWKhTHvrwqogOAhHOslaUQ==
MIME-Version: 1.0
X-Received: by 10.25.40.210 with SMTP id o201mr200709lfo.44.1456983093884;
Wed, 02 Mar 2016 21:31:33 -0800 (PST)
Received: by 10.25.86.68 with HTTP; Wed, 2 Mar 2016 21:31:33 -0800 (PST)
In-Reply-To: <87si089svf.fsf@redhat.com>
References:
<87si089svf.fsf@redhat.com>
Date: Wed, 2 Mar 2016 21:31:33 -0800
Message-ID:
Subject: Re: [pcp] GSOC 2016 idea for PCP
From: Charles Cui
X-ASG-Orig-Subj: Re: [pcp] GSOC 2016 idea for PCP
To: Lukas Berk
Cc: pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=001a11411b8ac13f4d052d1e4fd3
X-Barracuda-Connect: mail-lb0-f171.google.com[209.85.217.171]
X-Barracuda-Start-Time: 1456983095
X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256
X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, DKIM_SIGNED, DKIM_VERIFIED, HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27529
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header
-0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes
verification
0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature
0.00 HTML_MESSAGE BODY: HTML included in message
--001a11411b8ac13f4d052d1e4fd3
Content-Type: text/plain; charset=UTF-8
Hi Lukas,
Thanks for your reply! To answer your question. Yes, I have been
selected in GSoC 2014
and successfully complete the project. Also thanks for providing such a
great document.
I will read it carefully. By the way, is it enough to familiar with the
documents for the current phase?
Let me know if you want some initial work to be done for this period.
Thanks, Charles.
2016-03-02 18:51 GMT-08:00 Lukas Berk :
> Hi Charles,
>
> Charles Cui writes:
> [...]
> > This is Charles Cui from Tsinghua University, China. I am pretty
> > interested in the ideas listed in GSOC 2016 pages for PCP, especially
> > the idea of "performance metric extraction using go language agents".
> > As a PhD student in Tsinghua, I used C for roughly 4 years. At the
> > same time, I have golang experience for the production code. Besides,
> > I speak fluent English and am available the full summer. I am not sure
> > whether I can join in the development of PCP? In case my project is
> > selected by others, I am able to pick others.
>
> Thanks for your interest in PCP! Currently, no proposed project has
> been taken yet (official applications start on March 14), so no
> worries there. It's great to see that you've subscribed to the
> mailing list, we've also got a irc channel on freenode.net (#pcp) if
> you prefer that form of communication.
>
> Sounds like you've got some relevant programming language experience,
> have you contributed to any open source projects, or Google Summer of
> Code before?
>
> A few more details about the task itself:
>
> So, from a metrics collection standpoint, Performance Co-Pilot has two
> components. The first is the pmcd (1) (Performance Metrics Collector
> Daemon), which orchestrates client requests, and gathers the requested
> metrics from the individual pmdas (Performance Metric Domain Agents).
> Each of which define and collect a certain subset of metrics from the
> system in question. The pmcd is written in C, and currently we have
> pmdas (the agents), written in C/C++, perl, and python. We're looking
> to extend pmdas to Golang as well, which will include writing some of
> the standard pmda functions, as well as libraries, to interface between
> pmdas written in Golang, to the C level apis and libraries which
> currently exist.
>
> If you're particularly keen to read more about the technical
> implementation details within PCP, we've got plenty of documentation on
> the website[1], specifically the Programmer's Guide[2] (Chapter Two is
> specifically for writing a pmda).
>
> Please feel free to reach out with any more specific questions you might
> have, regardless if it's about the task, or the PCP community as a whole.
>
> Cheers,
>
> Lukas
>
> [1] - http://pcp.io/documentation.html
> [2] - http://pcp.io/books/PCP_PG/pdf/PCP-3-pcp-programmers-guide-en-US.pdf
>
>
--001a11411b8ac13f4d052d1e4fd3
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Hi Lukas,=C2=A0
=C2=A0 =C2=A0Thanks for=
your reply! To answer your question. Yes, I have been selected in GSoC 201=
4=C2=A0
and successfully complete the project. Also thanks for pr=
oviding such a great document.
I will read it carefully. By the w=
ay, is it enough to familiar with the documents for the current phase?=C2=
=A0
Let me know if you want some initial work to be done for this=
period.
Thanks, Charles.
2016-03-02 18:51 GMT-08:00 Luk=
as Berk
<lberk@redhat.com>:
Hi Charles,
Charles Cui <charles.cui198=
4@gmail.com> writes:
[...]
> This is Charles Cui from Tsinghua University, China. =
I am pretty
> interested in the ideas listed in GSOC 2016 pages for PCP, especially<=
br>
> the idea of "performance metric extraction using go language agen=
ts".
> As a PhD student in Tsinghua, I used C for roughly 4 years. At the
> same time, I have golang experience for the production code. Besides,<=
br>
> I speak fluent English and am available the full summer. I am not sure=
> whether I can join in the development of PCP? In case my project is
> selected by others, I am able to pick others.
Thanks for your interest in PCP!=C2=A0 Currently, no proposed projec=
t has
been taken yet (official applications start on March 14), so no
worries there.=C2=A0 It's great to see that you've subscribed to th=
e
mailing list, we've also got a irc channel on freenode.net (#pcp) if
you prefer that form of communication.
Sounds like you've got some relevant programming language experience,
have you contributed to any open source projects, or Google Summer of
Code before?
A few more details about the task itself:
So, from a metrics collection standpoint, Performance Co-Pilot has two
components.=C2=A0 The first is the pmcd (1) (Performance Metrics Collector<=
br>
Daemon), which orchestrates client requests, and gathers the requested
metrics from the individual pmdas (Performance Metric Domain Agents).
Each of which define and collect a certain subset of metrics from the
system in question.=C2=A0 The pmcd is written in C, and currently we have
pmdas (the agents), written in C/C++, perl, and python.=C2=A0 We're loo=
king
to extend pmdas to Golang as well, which will include writing some of
the standard pmda functions, as well as libraries, to interface between
pmdas written in Golang, to the C level apis and libraries which
currently exist.
If you're particularly keen to read more about the technical
implementation details within PCP, we've got plenty of documentation on=
the website[1], specifically the Programmer's Guide[2] (Chapter Two is<=
br>
specifically for writing a pmda).
Please feel free to reach out with any more specific questions you might
have, regardless if it's about the task, or the PCP community as a whol=
e.
Cheers,
Lukas
[1] - http://pcp.io/documentation.html
[2] - http://pcp.io/books/PCP_PG=
/pdf/PCP-3-pcp-programmers-guide-en-US.pdf
--001a11411b8ac13f4d052d1e4fd3--
From lberk@redhat.com Thu Mar 3 09:46:30 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
by oss.sgi.com (Postfix) with ESMTP id 35D277CA1
for ; Thu, 3 Mar 2016 09:46:30 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
by relay3.corp.sgi.com (Postfix) with ESMTP id ADB5FAC003
for ; Thu, 3 Mar 2016 07:46:26 -0800 (PST)
X-ASG-Debug-ID: 1457019982-04cb6c58311565b0001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id QzPCBzP7PVhnVfeB (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Thu, 03 Mar 2016 07:46:22 -0800 (PST)
X-Barracuda-Envelope-From: lberk@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22])
by mx1.redhat.com (Postfix) with ESMTPS id 12A68804EF;
Thu, 3 Mar 2016 15:46:22 +0000 (UTC)
Received: from toium (vpn-62-225.rdu2.redhat.com [10.10.62.225])
by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u23FkKUp029681
(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO);
Thu, 3 Mar 2016 10:46:21 -0500
From: Lukas Berk
To: Charles Cui
Cc: pcp@oss.sgi.com
Subject: Re: [pcp] GSOC 2016 idea for PCP
References:
<87si089svf.fsf@redhat.com>
X-ASG-Orig-Subj: Re: [pcp] GSOC 2016 idea for PCP
Date: Thu, 03 Mar 2016 10:46:20 -0500
In-Reply-To:
(Charles Cui's message of "Wed, 2 Mar 2016 21:31:33 -0800")
Message-ID: <874mcntvhv.fsf@redhat.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1457019982
X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384
X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
Hi Charles,
Charles Cui writes:
[...]
> Thanks for your reply! To answer your question. Yes, I have been
> selected in GSoC 2014
> and successfully complete the project. Also thanks for providing such
> a great document.
Nice! Was that the Plan9 project?
> I will read it carefully. By the way, is it enough to familiar with
> the documents for the current phase?
> Let me know if you want some initial work to be done for this period.
I would also recommend trying to build and install pcp from source[1] to
start, and perhaps even running the testsuite. What OS do you typically
use? (if possible, I would suggest starting with a popular linux distro,
it's a bit more simple to setup the build dependencies).
Check out the INSTALL file, specifically the 'B. Building From Source'
section, where I'd recommend the Makepkgs approach. Please feel free to
let me know if you run into any issues at all!
Cheers,
Lukas
[1] - git://pcp.io/pcp/pcp.git
From alec@alectenharmsel.com Thu Mar 3 11:39:01 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=T_DKIM_INVALID autolearn=ham
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
by oss.sgi.com (Postfix) with ESMTP id E59417CA1
for ; Thu, 3 Mar 2016 11:39:01 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
by relay2.corp.sgi.com (Postfix) with ESMTP id C31D0304039
for ; Thu, 3 Mar 2016 09:38:58 -0800 (PST)
X-ASG-Debug-ID: 1457026731-04bdf002ed17eb00001-S8gJnT
Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by cuda.sgi.com with ESMTP id h7Fr7JXIsyFSCfFY (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Thu, 03 Mar 2016 09:38:52 -0800 (PST)
X-Barracuda-Envelope-From: alec@alectenharmsel.com
X-Barracuda-Apparent-Source-IP: 66.111.4.25
Received: from compute2.internal (compute2.nyi.internal [10.202.2.42])
by mailout.nyi.internal (Postfix) with ESMTP id C11272151F
for ; Thu, 3 Mar 2016 12:38:51 -0500 (EST)
Received: from frontend1 ([10.202.2.160])
by compute2.internal (MEProxy); Thu, 03 Mar 2016 12:38:51 -0500
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=
messagingengine.com; h=content-type:date:from:in-reply-to
:message-id:mime-version:references:subject:to:x-sasl-enc
:x-sasl-enc; s=smtpout; bh=swTjcOqKzg7SWlGXkgKtZiEnZqE=; b=LRl6h
Cpve4r2UGaNtzHAd4l3Ii4zVEtV0HnQb4SuShG0uO5KRCzKBrMimmoT8X0dMS/bX
ZfPtNSqY8dK0GOUHL9hX1K1iQMJnEj49/9w3jfQ3GVooIZyoakVW3P9HiVfM93/U
jOvRu7YeCbS6SPBH6UQGFCWO6N9jEty1KZO4E8=
X-Sasl-enc: VNfjY10aQPh9/y95vzTLwk2feJN+/x4UanBDahdthcRo 1457026731
Received: from apio.adsroot.itcs.umich.edu (0587396917.wireless.umich.net [35.2.247.53])
by mail.messagingengine.com (Postfix) with ESMTPA id 41A4AC00014;
Thu, 3 Mar 2016 12:38:51 -0500 (EST)
Date: Thu, 3 Mar 2016 12:38:49 -0500
From: Alec Ten Harmsel
To: pcp@oss.sgi.com
Subject: Re: support for elasticsearch 2 metrics
Message-ID: <20160303173849.GC904@apio.adsroot.itcs.umich.edu>
X-ASG-Orig-Subj: Re: support for elasticsearch 2 metrics
References: <20160219145514.GB726@apio.adsroot.itcs.umich.edu>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="c3bfwLpm8qysLVxt"
Content-Disposition: inline
In-Reply-To: <20160219145514.GB726@apio.adsroot.itcs.umich.edu>
User-Agent: Mutt/1.5.24 (2015-08-30)
X-Barracuda-Connect: out1-smtp.messagingengine.com[66.111.4.25]
X-Barracuda-Start-Time: 1457026732
X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384
X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=DKIM_SIGNED, DKIM_VERIFIED
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27541
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
-0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes
verification
0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature
--c3bfwLpm8qysLVxt
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
On Fri, Feb 19, 2016 at 09:55:14AM -0500, Alec Ten Harmsel wrote:
> Hi,
>
> The ElasticSearch PMDA was mostly written in 2011-2013, and now the ES
> REST API has changed. Specifically, some of the URLs have changed:
>
> * _cluster/nodes/stats -> _nodes/stats
> * _cluster/nodes -> _nodes
There were plenty more changes than that. I have finally got around to
updating the PMDA.
I added a function, es_rest_version_internal(), that translates the
ElasticSearch (ES) version to an internal version (an integer) for ease
of use. es_rest_version_internal() is used in a few places to update
behavior for newer versions of ES, and keep behavior for older versions
of ES. I have been testing on my machine with ES 0.20.6 and 1.7.4.
The patch is attached for review; I can use GitHub or anything else that
is the preferred method of sharing patches.
Alec
--c3bfwLpm8qysLVxt
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="new_es_support.patch"
diff --git a/src/pmdas/elasticsearch/pmdaelasticsearch.pl b/src/pmdas/elasticsearch/pmdaelasticsearch.pl
index a66a347..ff99769 100755
--- a/src/pmdas/elasticsearch/pmdaelasticsearch.pl
+++ b/src/pmdas/elasticsearch/pmdaelasticsearch.pl
@@ -12,6 +12,17 @@
# for more details.
#
+# NOTE: This PMDA attempts to support all versions of ElasticSearch. The
+# ElasticSearch REST API changed from <=0.20 to >=0.90. We internally version
+# the ES REST API, since the upstream developers did not version it.
+#
+# | Internal version | ElasticSearch version range |
+# | 1 | [0, 0.20.6] |
+# | 2 | [0.90.0, 2.x] |
+#
+# The ElasticSearch version can be mapped to an internal version by running
+# es_rest_version_internal().
+
use strict;
use warnings;
use JSON;
@@ -135,13 +146,20 @@ sub es_rewrite_cluster_state
foreach my $index_key (keys %$indices) {
# Go over each setting key and transpose what the key name is called
my $settings = $indices->{$index_key}->{'settings'};
- foreach my $settings_key (keys %$settings) {
- # Convert keys like "index.version.created" to "version_created"
- my $transformed_key = $settings_key;
- $transformed_key =~ s/index\.//;
- $transformed_key =~ s/\./_/g;
- $settings->{$transformed_key} = $settings->{$settings_key};
- }
+
+ if (es_rest_version_internal() == 1) {
+ foreach my $settings_key (keys %$settings) {
+ # Convert keys like "index.version.created" to "version_created"
+ my $transformed_key = $settings_key;
+ $transformed_key =~ s/index\.//;
+ $transformed_key =~ s/\./_/g;
+ $settings->{$transformed_key} = $settings->{$settings_key};
+ }
+ } else {
+ $indices->{$index_key}->{'settings'} = $settings->{'index'};
+ $settings = $indices->{$index_key}->{'settings'};
+ $settings->{'version_created'} = $settings->{'version'}->{'created'};
+ }
}
}
@@ -159,7 +177,14 @@ sub es_refresh_cluster_state
sub es_refresh_cluster_nodes_stats_all
{
- my $content = es_agent_get($baseurl . "_cluster/nodes/stats?all");
+ my $url = undef;
+ if (es_rest_version_internal() == 1) {
+ $url = "_cluster/nodes/stats?all";
+ } else {
+ $url = "_nodes/stats?all";
+ }
+
+ my $content = es_agent_get($baseurl . $url);
if (defined($content)) {
$es_nodestats = decode_json($content);
es_data_node_instances($es_nodestats->{'nodes'});
@@ -170,7 +195,14 @@ sub es_refresh_cluster_nodes_stats_all
sub es_refresh_cluster_nodes_all
{
- my $content = es_agent_get($baseurl . "_cluster/nodes?all");
+ my $url = undef;
+ if (es_rest_version_internal() == 1) {
+ $url = "_cluster/nodes?all";
+ } else {
+ $url = "_nodes?all";
+ }
+
+ my $content = es_agent_get($baseurl . $url);
if (defined($content)) {
$es_nodes = decode_json($content);
es_data_node_instances($es_nodes->{'nodes'});
@@ -190,7 +222,11 @@ sub es_refresh_stats_search
my $content = es_agent_get($baseurl . "_stats/search");
if (defined($content)) {
$es_searchstats = decode_json($content);
- es_search_instances($es_searchstats->{'_all'}->{'indices'});
+ if (es_rest_version_internal() == 1) {
+ es_search_instances($es_searchstats->{'_all'}->{'indices'});
+ } else {
+ es_search_instances($es_searchstats->{'indices'});
+ }
} else {
$es_searchstats = undef;
}
@@ -236,7 +272,12 @@ sub es_lookup_search
{
my ($json, $inst) = @_;
my $searchID = $search_instance_ids[($inst*2)+1];
- return $json->{'_all'}->{'indices'}->{$searchID};
+
+ if (es_rest_version_internal() == 1) {
+ return $json->{'_all'}->{'indices'}->{$searchID};
+ } else {
+ return $json->{'indices'}->{$searchID};
+ }
}
sub es_lookup_index
@@ -246,6 +287,22 @@ sub es_lookup_index
return $json->{'metadata'}->{'indices'}->{$indexID};
}
+# Returns our internal REST version number
+sub es_rest_version_internal
+{
+ my $es_version = es_value($es_root, ['version', 'number']);
+
+ if ($es_version =~ /^0\.[0-9]+\.[0-9]+$/ and
+ (not $es_version =~ /^0\.90/)) {
+ return 1;
+ } else {
+ return 2;
+ }
+
+ # Assume a recent version of ES
+ return 2;
+}
+
# iterate over metric-name components, performing hash lookups as we go.
sub es_value
{
--c3bfwLpm8qysLVxt--
From charles.cui1984@gmail.com Thu Mar 3 12:18:43 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.3 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,
FREEMAIL_FROM,HTML_MESSAGE,T_DKIM_INVALID autolearn=no version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
by oss.sgi.com (Postfix) with ESMTP id DB7047CA1
for ; Thu, 3 Mar 2016 12:18:43 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
by relay1.corp.sgi.com (Postfix) with ESMTP id AE7E78F8035
for ; Thu, 3 Mar 2016 10:18:40 -0800 (PST)
X-ASG-Debug-ID: 1457029117-04cbb00b7218f440001-S8gJnT
Received: from mail-lb0-f170.google.com (mail-lb0-f170.google.com [209.85.217.170]) by cuda.sgi.com with ESMTP id 7FVX6XOyxzbAoVPc (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Thu, 03 Mar 2016 10:18:38 -0800 (PST)
X-Barracuda-Envelope-From: charles.cui1984@gmail.com
X-Barracuda-Apparent-Source-IP: 209.85.217.170
Received: by mail-lb0-f170.google.com with SMTP id x1so33635943lbj.3
for ; Thu, 03 Mar 2016 10:18:38 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20120113;
h=mime-version:in-reply-to:references:date:message-id:subject:from:to
:cc;
bh=vjbfTapS5PamA9ea0GYFL3RSS725EKkWnPKHrMjBOb4=;
b=DDIMefbD0bgJXsZjBQl2xE+dgANnDpBv4PSyoPWK2gOlLNTufQ3QuWd/pg+FLNFK5Y
K/WhsgO6ALJLsuoSaj0Jb/jeddBRlJAtjvhnnZ4tOIFd1xtMTcQfMrX9YEutbMObZLB3
kJRtLesw3XmGobqRW+jL6rVlNm+JF+KtCj/Gw0wxtwdOa1DbvhDUDeW3EynTFCKRXH6H
RvnHQAZ3+D8/dZsSlxoQvb+tDX19DzvUcYvqZK+qxOFnYKqgzA9YmVDyYlyH6tlqpfl7
ozz4ZmjtiV/qttc+azFTshkO+L1N5zQ8btRb/mV9OtTzOf2kD4yVk3nsqj0GS9wnm6K4
GnDQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20130820;
h=x-gm-message-state:mime-version:in-reply-to:references:date
:message-id:subject:from:to:cc;
bh=vjbfTapS5PamA9ea0GYFL3RSS725EKkWnPKHrMjBOb4=;
b=FHXH/wlW3EysareSkGaABUvbW0641hQUnMofY5otcJ3X6NnagacCcsAy5aJp5VHcWx
eqL5d2tKL7fcu30WEIllIJAt9vofiuVM2ai/g9D1IV/jw1BSat4Fb+9QoHznJe1JIzt+
ZDe6/LHY77U1ITyqLs/soMMbv8eN4EMjJpK3I74oLYxA94jOlNUvqXP1fMqO/fJirOnr
Inwdr9EKd93Kfpt9cpNkj5XXkeBRfUr/5j7+XA/gvwDl8MfaN0DwTbKHL+DHZRxDI7Vg
E3APlj3JF/iJiCniBj7ClyHLmdES6Wzx5Ss5OcuwrJF/5u6X3IjWQ+gxgK6H2kmDkaKl
tpqw==
X-Gm-Message-State: AD7BkJJbQ5ImPa4uvuA12afSBY9rNvMsj//7Y4agIVnSsw13hc5q6hDKAv1GLD+srIPY7Ffgh/3lVQyi16lv0w==
MIME-Version: 1.0
X-Received: by 10.25.154.65 with SMTP id c62mr1669427lfe.54.1457029116787;
Thu, 03 Mar 2016 10:18:36 -0800 (PST)
Received: by 10.25.86.68 with HTTP; Thu, 3 Mar 2016 10:18:36 -0800 (PST)
In-Reply-To: <874mcntvhv.fsf@redhat.com>
References:
<87si089svf.fsf@redhat.com>
<874mcntvhv.fsf@redhat.com>
Date: Thu, 3 Mar 2016 10:18:36 -0800
Message-ID:
Subject: Re: [pcp] GSOC 2016 idea for PCP
From: Charles Cui
X-ASG-Orig-Subj: Re: [pcp] GSOC 2016 idea for PCP
To: Lukas Berk
Cc: pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=001a114012b4ef05ff052d290660
X-Barracuda-Connect: mail-lb0-f170.google.com[209.85.217.170]
X-Barracuda-Start-Time: 1457029117
X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256
X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, DKIM_SIGNED, DKIM_VERIFIED, HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27542
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header
-0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes
verification
0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature
0.00 HTML_MESSAGE BODY: HTML included in message
--001a114012b4ef05ff052d290660
Content-Type: text/plain; charset=UTF-8
2016-03-03 7:46 GMT-08:00 Lukas Berk :
> Hi Charles,
>
> Charles Cui writes:
> [...]
> > Thanks for your reply! To answer your question. Yes, I have been
> > selected in GSoC 2014
> > and successfully complete the project. Also thanks for providing such
> > a great document.
>
> Nice! Was that the Plan9 project?
>
> > I will read it carefully. By the way, is it enough to familiar with
> > the documents for the current phase?
> > Let me know if you want some initial work to be done for this period.
>
> I would also recommend trying to build and install pcp from source[1] to
> start, and perhaps even running the testsuite. What OS do you typically
> use? (if possible, I would suggest starting with a popular linux distro,
> it's a bit more simple to setup the build dependencies).
>
> Check out the INSTALL file, specifically the 'B. Building From Source'
> section, where I'd recommend the Makepkgs approach. Please feel free to
> let me know if you run into any issues at all!
>
Great! I will start to run PCP locally on ubuntu. I am sure there will be
some issues for this project from me. I will let the community know.
>
> Cheers,
>
> Lukas
>
> [1] - git://pcp.io/pcp/pcp.git
>
--001a114012b4ef05ff052d290660
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
--001a114012b4ef05ff052d290660--
From myllynen@redhat.com Thu Mar 3 12:42:32 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.5 required=5.0 tests=SUBJ_ALL_CAPS autolearn=no
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
by oss.sgi.com (Postfix) with ESMTP id 7F7777CA1
for ; Thu, 3 Mar 2016 12:42:32 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
by relay3.corp.sgi.com (Postfix) with ESMTP id EDBB4AC001
for ; Thu, 3 Mar 2016 10:42:28 -0800 (PST)
X-ASG-Debug-ID: 1457030541-04cbb00b7018fd80001-S8gJnT
Received: from mail-wm0-f42.google.com (mail-wm0-f42.google.com [74.125.82.42]) by cuda.sgi.com with ESMTP id ELUGe1Wgw9uXiGne (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Thu, 03 Mar 2016 10:42:22 -0800 (PST)
X-Barracuda-Envelope-From: myllynen@redhat.com
X-Barracuda-Apparent-Source-IP: 74.125.82.42
Received: by mail-wm0-f42.google.com with SMTP id p65so2488523wmp.0
for ; Thu, 03 Mar 2016 10:42:22 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20130820;
h=x-gm-message-state:reply-to:to:from:subject:organization:message-id
:date:user-agent:mime-version:content-transfer-encoding;
bh=fTe/U3YewuTMk0FWReDey6TP1YIWGjYCnzXsdYg70fw=;
b=DGsWtk4WBsBDDTMsxy76ClbQ5sF5NFhRPPuHIohcH5HPsKnQP3vjVQTAOUcfojGOLH
4ogbERddoTeXjFTGN+wDIQt/Az70uVSOYFrSgBYq3mQYb6kJ9HkuJUs1lkKn9wn2AY5v
8jzirp4Rx0dMCH1Dw4f0GCgqcyFHtkwKpgMSWF8NfTUniqJGJBS7J6hlqh1YrwUj4b9r
IelpfTv6yrpbSNi6jXsG2EJUU/FYnyzCcZbKYRIk1crjYOMyEP3F7ur9JpE88836ppDi
8rh5omVtAcu3+sCQ9AkZIRbmxKkRBBLM0BwFeGdNmH4w0vefOgB+CbRBB6ylxMg+lsgQ
eQuQ==
X-Gm-Message-State: AD7BkJLDdDcaKtVrtFc8Sf/vK4Q+rZtYMx+4h+mUyMhKHXuOUM9P7k5HQ5EpZPTTutT+4fk5
X-Received: by 10.28.172.132 with SMTP id v126mr621027wme.28.1457030541221;
Thu, 03 Mar 2016 10:42:21 -0800 (PST)
Received: from ?IPv6:2001:14bb:100:607:402:1fcc:a508:3? (dyfyypb4fyks85lkkyybt-4.rev.dnainternet.fi. [2001:14bb:100:607:402:1fcc:a508:3])
by smtp.gmail.com with ESMTPSA id z65sm3887378wmg.1.2016.03.03.10.42.20
for
(version=TLSv1/SSLv3 cipher=OTHER);
Thu, 03 Mar 2016 10:42:20 -0800 (PST)
Reply-To: Marko Myllynen
To: pcp developers
From: Marko Myllynen
Subject: PCP JMX PMDA
Organization: Red Hat
X-ASG-Orig-Subj: PCP JMX PMDA
Message-ID: <56D8858A.3020407@redhat.com>
Date: Thu, 3 Mar 2016 20:42:18 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
Thunderbird/38.6.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Barracuda-Connect: mail-wm0-f42.google.com[74.125.82.42]
X-Barracuda-Start-Time: 1457030542
X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256
X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 2.38
X-Barracuda-Spam-Status: No, SCORE=2.38 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_RULE_7580D, SUBJ_ALL_CAPS, SUBJ_ALL_CAPS_2
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27542
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
0.01 SUBJ_ALL_CAPS Subject is all capitals
0.75 BSF_RULE_7580D Custom Rule 7580D
1.62 SUBJ_ALL_CAPS_2 SUBJ_ALL_CAPS_2
Hi!
PCP has very complete coverage for system and supporting applications /
infrastructure metrics (like containers, 389 Directory Server, KVM,
Oracle, PostgreSQL, etc.) but there are lots of places where Java
performance metrics would be essential to have in the mix as well.
To fill this gap for PCP, I've written PCP JMX PMDA, below is a quick
illustration what kind of metrics this PMDA plugin provides:
$ pminfo jmx | wc -l
274
$ pminfo jmx | grep -Ei '(garbagecollector|heap|memory)' | wc -l
150
$ pminfo jmx.threading
jmx.threading.threadallocatedmemorysupported
jmx.threading.peakthreadcount
jmx.threading.threadcputimeenabled
jmx.threading.totalstartedthreadcount
jmx.threading.threadcputimesupported
jmx.threading.currentthreadcputimesupported
jmx.threading.threadcount
jmx.threading.allthreadids
jmx.threading.threadcontentionmonitoringsupported
jmx.threading.currentthreadcputime
jmx.threading.threadcontentionmonitoringenabled
jmx.threading.synchronizerusagesupported
jmx.threading.currentthreadusertime
jmx.threading.daemonthreadcount
jmx.threading.objectmonitorusagesupported
jmx.threading.threadallocatedmemoryenabled
$ pminfo -dfm jmx.threading.threadcount
jmx.threading.threadcount PMID: 498.0.153
Data Type: 32-bit int InDom: 498.0 0x7c800000
Semantics: instant Units: count
inst [0 or "localhost:9010"] value 11
inst [1 or "localhost:9012"] value 14
$ pminfo -dfm jmx.runtime.systemproperties.sun_java_command
jmx.runtime.systemproperties.sun_java_command PMID: 498.0.106
Data Type: string InDom: 498.0 0x7c800000
Semantics: instant Units: none
inst [0 or "localhost:9010"] value "JavaTest"
inst [1 or "localhost:9012"] value "JavaTest"
$
Below are some design principles and implementation notes, let me know
what you think, could this be considered to merged to PCP?
The design goals were to have zero Java code changes for monitored
applications, preferably also zero configuration changes. Target was
both local Java applications and remote Java apps over the network.
Since there's a wide range of metrics available and use cases are wildly
different, the user should have full control on what metrics to collect,
if possible also from custom performance metrics providers. Lastly, the
implementation itself should also have minimum number of dependencies to
external libraries to keep it as portable as possible.
On the implementation side JMX is the standard API for management and
monitoring resources [1] so it was the natural choice to be used for
collection the metrics. Since we're talking Java/JMX, it's best to be
done from a Java application. I checked some existing alternatives but
they were either non-native / not available for many distributions (like
jmx4perl) or not actively maintained and/or somewhat limited (various
command line JMX utilities), especially when it comes to attribute
filtering and output formatting results. So writing a new one from
scratch specifically for PCP was IMHO warranted. This is now called
PCPJMXConnector.
OTOH, PCP does not have Java PMDA API so a counterpart component in a
supported language would be needed. I chose to use Perl as it's pretty
good in extraction and reporting and also because the Perl PMDA API is
slightly more complete than the Python PMDA API (e.g., pipe support is
available only with Perl).
More practically speaking, one the issues to take into consideration
from early on was the fact that it's a known fact that under extreme
load the JMX requests might take several seconds to complete - otherwise
no such a big deal but since PMCD is quite trigger happy with slowly
responding PMDAs the JMX PMDA must allow JMX requests to complete while
still keeping PMCD at bay (the worst case scenario to avoid is that when
under extreme load, perhaps due to application issues, either the PMDA
gets killed or is unable to collect metrics when they are most needed).
To achieve this, the Java side uses connection caching and threading to
be as efficient as possible and the Perl side reads the results from a
pipe (or a tailed file, if configured) as they come and only give them
to PMCD once a consistent set of fresh metrics are available, otherwise
providing instant replies with the most recent complete results.
The Perl implementation should be pretty easy to read for anyone
familiar with PCP PMDAs, the one thing worth noting is that it support
three different modes of communication with the Java application (with
the cost of only few extra lines): in-pmda piping (the default, works
well), libpcp piping (hopefully works after the yesterday's fix from
Ken), and tailing from a file (works ok, uses a local file which can be
later discarded/stored as needed).
The Java side outputs CSV - it would be easy to extend it to support
other formats like JSON as well (but I'm not sure how well it would work
with the JSON PMDA then, perhaps something that could be tested in the
future).
The Java side has two alternatives for collecting the metrics over JMX.
The first one is to use the Attach API [2] which will also automatically
start to management agent on the target JVM if needed to collect data
from MBeans [3]. This mode can be used to monitor any running JVM on the
local system, filtering for local JVMs is supported as well. Due to this
mode, one external requirement for the Java code is present, namely the
standard tools.jar which is part of Java SDK (included in the -devel
package on many distributions).
The second alternative is to use host:port type specifications to
monitor only a selected list of JVMs, local or remote. In this case the
JVM needs to be started with the management already enabled [4].
The way to define the metrics to be collected is fully dynamic, I've
tested the filtering configurations against the standard MBeans but any
custom MBean providing supported data types should work as well. The
standard JMX filtering [5] can be used but that might be a bit
inconvenient for some, thus there's a also a PCP JMX Connector specific
filtering alternative (which can even be combined with the standard JMX
filtering to certain extent):
# Question mark (?) separated list of beans. Beans may optionally be
# followed by a list attributes as a pipe (|) separated list. List mode
# lists matching attributes. Note that the attribute filter matches
# innermost attributes only, like the Vm* ones in the example below.
#
# Example:
java.lang:type=Runtime|VmName|VmVendor|VmVersion?java.lang:type=OperatingSystem
# Default:
java.lang:type=Memory?java.lang:type=Threading?java.lang:type=Runtime|sun.java.command
As the comment from the configuration file example states, the attribute
filters match only the innermost (I'd say the "real") attributes -
meaning that instead of fetching 50+ irrelevant Runtime/SystemProperties
attributes in a higher level container data structure only the
intesresting attributes can be selected.
There's also a list mode available so that one can examine what
attributes (metrics) are available and what the current filtering
configuration is providing.
The code is currently in my Fedora people area (no public git repo
available, sorry), as said the tools.jar is the only requirement so
using -cp /etc/alternatives/java_sdk/lib/tools.jar:. or such with
java/javac should be enough:
https://myllynen.fedorapeople.org/pcp-jmx/
This is 1.6 / 1.7 / 1.8 compatible, changes to support 1.6 and 1.7 were
reasonable and didn't affect the code much on the whole.
Lastly I think I could mention that I've seen some of the earlier PCP
list discussions about Parfait and Jolokia based instrumentation. They
are certainly interesting alternatives and might be wanted by some and
could be even more suitable than this JMX PMDA in some cases but since
the standard JMX metrics are enough for many, I think if those Parfait
and Jolokia alternatives materialize in the PCP upstream one day then we
can simply have different alternatives (PMDAs) to cover different use
cases. However, I'm currently interested only in this JMX PMDA.
1) https://docs.oracle.com/javase/8/docs/technotes/guides/jmx/index.html
2) https://docs.oracle.com/javase/8/docs/jdk/api/attach/spec/index.html
3) https://docs.oracle.com/javase/tutorial/jmx/mbeans/standard.html
4) To start a JVM with remote JMX enable use something like:
java -Dcom.sun.management.jmxremote.port=9012
-Dcom.sun.management.jmxremote.rmi.port=9012
-Dcom.sun.management.jmxremote.local.only=false
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false ...
5)
https://docs.oracle.com/javase/8/docs/api/javax/management/ObjectName.html
Thanks,
--
Marko Myllynen
From kenj@kenj.com.au Thu Mar 3 20:06:44 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
by oss.sgi.com (Postfix) with ESMTP id A9C7C7CA1
for ; Thu, 3 Mar 2016 20:06:44 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
by relay3.corp.sgi.com (Postfix) with ESMTP id 31A0EAC002
for ; Thu, 3 Mar 2016 18:06:41 -0800 (PST)
X-ASG-Debug-ID: 1457057198-04bdf002ed18cee0001-S8gJnT
Received: from ipmail05.adl6.internode.on.net (ipmail05.adl6.internode.on.net [150.101.137.143]) by cuda.sgi.com with ESMTP id cSTysLCldBxm4fHu for ; Thu, 03 Mar 2016 18:06:38 -0800 (PST)
X-Barracuda-Envelope-From: kenj@kenj.com.au
X-Barracuda-Apparent-Source-IP: 150.101.137.143
Received: from ppp121-44-23-103.lns20.syd4.internode.on.net (HELO kenj) ([121.44.23.103])
by ipmail05.adl6.internode.on.net with ESMTP; 04 Mar 2016 12:36:07 +1030
Received: by kenj (Postfix, from userid 1000)
id 60BE99FEEB; Fri, 4 Mar 2016 13:06:01 +1100 (AEDT)
To: pcp@oss.sgi.com
Subject: QA Status
Message-Id: <20160304020601.60BE99FEEB@kenj>
X-ASG-Orig-Subj: QA Status
Date: Fri, 4 Mar 2016 13:06:01 +1100 (AEDT)
From: kenj@kenj.com.au (Ken McDonell)
X-Barracuda-Connect: ipmail05.adl6.internode.on.net[150.101.137.143]
X-Barracuda-Start-Time: 1457057198
X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27558
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
Details for the most commonly failing tests are available from
ftp://oss.sgi.com/projects/pcp/download/qa-status/
=== Synopsis ===
QA Status from Ken McDonell's QA Farm in Melbourne
Summary: 24025 run, 62 failed (0.26%)
QA Test Fail Failure Signatures (number of different failure patterns)
878 11 6 3 1 1
1038 5 3 1 1
983 4 3 1
1069 3 3
1062 3 3
1031 3 2 1
880 3 3
823 3 1 1 1
756 3 3
^ ^
| |
| +-- A B C ... means one failure signature occurred A times,
| another occurred B times, a third occurred C times, ...
+-- total number of failures for this test, equals A+B+C+...
Each tarball in this directory contains .out, .out.bad and possibly
.full files for all failing test cases where a test failed on at least
9% of the QA hosts.
The latest tarball is 20160304.tgz.
==== QA Summary ====
Date Run Pass Fail Nrun Host
2016-03-02 819 815 4 85|bo|bozo PCP 3.11.1 x86_64 Ubuntu 15.10
2016-03-04 794 791 3 110|bl|bozo-laptop PCP 3.11.1 i686 LinuxMint 15
2016-03-04 806 805 1 98|bv|bozo-vm PCP 3.11.1 x86_64 Debian 8.2
2016-03-04 704 699 5 157|gr|grundy PCP 3.11.0 ia64 SUSE SLES11 SP1
2016-03-04 823 822 1 81|00|vm00 PCP 3.11.1 x86_64 Ubuntu 12.04
2016-03-01 826 826 0 78|01|vm01 PCP 3.11.1 i686 Ubuntu 15.10
2016-03-01 812 810 2 92|02|vm02 PCP 3.11.1 i686 openSUSE 13.2
2016-03-01 848 846 2 56|03|vm03 PCP 3.11.1 x86_64 Fedora 21
2016-03-02 736 729 7 168|04|vm04 PCP 3.11.1 i586 CentOS 5.11
2016-03-02 822 818 4 82|05|vm05 PCP 3.11.1 x86_64 Gentoo 2.2
2016-03-02 61 61 0 4|06|vm06 PCP 3.11.1 amd64 FreeBSD 10.2-RELEASE
2016-03-02 804 800 4 100|07|vm07 PCP 3.11.1 x86_64 Debian 6.0.10
2016-02-29 846 845 1 58|08|vm08 PCP 3.11.1 x86_64 CentOS Linux7.2.1511
2016-02-29 61 61 0 4|09|vm09 PCP 3.11.1 i386 NetBSD 6.1.5
2016-03-01 61 61 0 4|10|vm10 PCP 3.11.1 i386 FreeBSD 9.3-RELEASE-p30
2016-03-02 803 800 3 101|11|vm11 PCP 3.11.1 i686 Debian 6.0.10
2016-03-02 844 843 1 60|12|vm12 PCP 3.11.1 i686 Fedora 22
2016-03-03 839 838 1 65|14|vm14 PCP 3.11.1 x86_64 CentOS6.7
No daily runs |15|vm15 PCP 3.9.1 x86_64 Slackware 13.37.0
2016-03-03 834 833 1 70|18|vm18 PCP 3.11.1 x86_64 LinuxMint 17.3
2016-03-03 814 812 2 90|19|vm19 PCP 3.11.1 x86_64 openSUSE 12.2
2016-03-03 816 815 1 88|20|vm20 PCP 3.11.1 x86_64 Ubuntu 13.04
2016-03-03 817 817 0 87|21|vm21 PCP 3.11.1 i686 Debian 7.9
2016-03-03 843 842 1 61|22|vm22 PCP 3.11.1 x86_64 Fedora 19
2016-03-03 842 841 1 62|23|vm23 PCP 3.11.1 i686 Fedora 20
2016-03-04 837 835 2 67|24|vm24 PCP 3.11.1 i686 openSUSE 13.1
2016-02-29 732 725 7 172|25|vm25 PCP 3.11.1 x86_64 CentOS 5.11
2016-03-01 846 844 2 58|26|vm26 PCP 3.11.1 x86_64 Fedora 21
2016-03-02 835 835 0 69|27|vm27 PCP 3.11.1 x86_64 Ubuntu 15.04
2016-03-03 841 840 1 63|28|vm28 PCP 3.11.1 x86_64 RHEL Server 6.7
2016-03-03 848 847 1 56|29|vm29 PCP 3.11.1 x86_64 RHEL Server 7.1
2016-02-29 844 844 0 60|30|vm30 PCP 3.11.1 x86_64 SUSE SLES12 SP0
2016-03-01 845 843 2 59|31|vm31 PCP 3.11.1 x86_64 Fedora 23
2016-03-03 61 59 2 4|32|vm32|vm32
2016-03-03 61 61 0 4|33|vm33 PCP 3.11.1 amd64 OpenBSD 5.8
Summary: 24025 run, 62 failed (0.26%)
==== QA Failure (X) Map ====
Host bo bl bv gr 00 02 03 04 05 07 08 11 12 14 18 19 20 22 23 24 25 26 28 29 31 32
Test %fail Test QA groups
878 32% X X X X X X X X X X X 878 pmda.pipe pmval
1038 15% X X X X X 1038 pmrep archive multi-archive
983 12% X X X X 983 pmie
756 9% X X X 756 pmda.perfevent
823 9% X X X 823 pmda.sample pmstore secure
880 9% X X X 880 pmrep python
1031 9% X X X 1031 pmda.slurm perl
1062 9% X X X 1062 pmrep python
1069 9% X X X 1069 pmrep python
169 6% X X 169 pmcd
666 6% X X 666 pmmgr slow
809 6% X X 809 archive multi-archive
810 6% X X 810 pmiostat archive multi-archive
1036 6% X X 1036 pmclient archive multi-archive
206 3% X 206 archive pmval
280 3% X 280 pmlogger logutil folio pmlc
436 3% X 436 archive pmdumplog
437 3% X 437 archive
Host bo bl bv gr 00 02 03 04 05 07 08 11 12 14 18 19 20 22 23 24 25 26 28 29 31 32
518 3% X 518 pcp flakey pmie
555 3% X 555 pmie
651 3% X 651 pmproxy
661 3% X 661 pmwebapi
662 3% X 662 pmwebapi pmda.sample
667 3% X 667 python
709 3% X 709 python
710 3% X 710 python pmimport
739 3% X 739 python getopt
892 3% X 892 pmcd libpcp
Host bo bl bv gr 00 02 03 04 05 07 08 11 12 14 18 19 20 22 23 24 25 26 28 29 31 32
From nscott@redhat.com Thu Mar 3 22:46:04 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: *
X-Spam-Status: No, score=1.0 required=5.0 tests=FRT_ADOBE2 autolearn=no
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
by oss.sgi.com (Postfix) with ESMTP id 6C23A7CA1
for ; Thu, 3 Mar 2016 22:46:04 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
by relay1.corp.sgi.com (Postfix) with ESMTP id 165748F8033
for ; Thu, 3 Mar 2016 20:46:01 -0800 (PST)
X-ASG-Debug-ID: 1457066754-04bdf002ec191720001-S8gJnT
Received: from mx6-phx2.redhat.com (mx6-phx2.redhat.com [209.132.183.39]) by cuda.sgi.com with ESMTP id GDcTTXLTiLvM3ONe (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Thu, 03 Mar 2016 20:45:54 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.39
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
by mx6-phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u244jspO027419;
Thu, 3 Mar 2016 23:45:54 -0500
Date: Thu, 3 Mar 2016 23:45:54 -0500 (EST)
From: Nathan Scott
Reply-To: Nathan Scott
To: Marko Myllynen
Cc: pcp developers
Message-ID: <675485289.27235420.1457066754101.JavaMail.zimbra@redhat.com>
In-Reply-To: <1240982544.27234304.1457065841902.JavaMail.zimbra@redhat.com>
Subject: [PATCH] initial pass at fixing zbxpcp.so for Zabbix v3
MIME-Version: 1.0
X-ASG-Orig-Subj: [PATCH] initial pass at fixing zbxpcp.so for Zabbix v3
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.64.48.181]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF44 (Linux)/8.0.6_GA_5922)
Thread-Topic: initial pass at fixing zbxpcp.so for Zabbix v3
Thread-Index: aNUGAh9rE6p9sMmax+al37ilCoy0Pg==
X-Barracuda-Connect: mx6-phx2.redhat.com[209.132.183.39]
X-Barracuda-Start-Time: 1457066754
X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384
X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.03
X-Barracuda-Spam-Status: No, SCORE=0.03 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_SA_TO_FROM_DOMAIN_MATCH, THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27560
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)...
0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header
0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient
Domain
Hi Marko,
This should fix the problematic interactions between zbxpcp and the
new version of Zabbix with the ABI/API break. I noticed the problem
structure (AGENT_RESULT) can actually be determined at runtime, so
we can (at least in theory) solve this via a single .so as we first
hoped. We don't ever use zbx_log_t (which also changed), and it is
only ever referenced via pointer, so I made that opaque and dropped
its definition entirely.
This switches between using v2/v3 interface based on presence or lack
of a named file. This is a bit suboptimal as it still involves some
manual user interaction, but its better than having multiple shared
libraries I think. If we could find some reliable way of detecting a
Zabbix 3 vs 2 install (?) at runtime, we'd be laughing. Something in
the environment, or a file we could test for? A symbol we could look
for with dlsym, even? Anyway, this will probably do for now.
I don't know whether we need to set the new AGENT_RESULT fields, i.e.
lastlogsize and mtime, in the zbxpcp.so v3 callback code - any idea?
cheers.
diff --git a/src/zabbix-agent/src/module.h b/src/zabbix-agent/src/module.h
index aaccc61..91c32d1 100644
--- a/src/zabbix-agent/src/module.h
+++ b/src/zabbix-agent/src/module.h
@@ -1,5 +1,5 @@
/*
-** Copyright (C) 2001-2015 Zabbix SIA
+** Copyright (C) 2001-2016 Zabbix SIA
**
** This program is free software; you can redistribute it and/or modify
** it under the terms of the GNU General Public License as published by
@@ -22,6 +22,8 @@ typedef __uint64_t zbx_uint64_t;
#define ZBX_MODULE_API_VERSION_ONE 1
+#define get_rkey(request) (request)->key
+#define get_rparams_num(request) (request)->nparam
#define get_rparam(request, num) ((request)->nparam > num ? (request)->params[num] : NULL)
/* flags for command */
@@ -49,17 +51,17 @@ typedef struct
}
AGENT_REQUEST;
-typedef struct
-{
- char *value;
- char *source;
- zbx_uint64_t lastlogsize;
- int timestamp;
- int severity;
- int logeventid;
- int mtime;
-}
-zbx_log_t;
+struct zbx_log;
+typedef struct zbx_log zbx_log_t;
+
+/* agent result types */
+#define AR_UINT64 0x01
+#define AR_DOUBLE 0x02
+#define AR_STRING 0x04
+#define AR_TEXT 0x08
+#define AR_LOG 0x10
+#define AR_MESSAGE 0x20
+#define AR_META 0x40
/* agent return structure */
typedef struct
@@ -74,15 +76,21 @@ typedef struct
/* null-terminated list of pointers */
zbx_log_t **logs;
}
-AGENT_RESULT;
+AGENT_RESULT_V2;
-/* agent result types */
-#define AR_UINT64 0x01
-#define AR_DOUBLE 0x02
-#define AR_STRING 0x04
-#define AR_TEXT 0x08
-#define AR_LOG 0x10
-#define AR_MESSAGE 0x20
+typedef struct
+{
+ zbx_uint64_t lastlogsize; /* meta information */
+ zbx_uint64_t ui64;
+ double dbl;
+ char *str;
+ char *text;
+ char *msg; /* possible error message */
+ zbx_log_t *log;
+ int type; /* flags: see AR_* above */
+ int mtime; /* meta information */
+}
+AGENT_RESULT_V3;
/* SET RESULT */
@@ -113,13 +121,6 @@ AGENT_RESULT;
)
/* NOTE: always allocate new memory for val! DON'T USE STATIC OR STACK MEMORY!!! */
-#define SET_LOG_RESULT(res, val) \
-( \
- (res)->type |= AR_LOG, \
- (res)->logs = (zbx_log_t **)(val) \
-)
-
-/* NOTE: always allocate new memory for val! DON'T USE STATIC OR STACK MEMORY!!! */
#define SET_MSG_RESULT(res, val) \
( \
(res)->type |= AR_MESSAGE, \
diff --git a/src/zabbix-agent/src/zbxpcp.c b/src/zabbix-agent/src/zbxpcp.c
index 5e20d4f..700630f 100644
--- a/src/zabbix-agent/src/zbxpcp.c
+++ b/src/zabbix-agent/src/zbxpcp.c
@@ -39,6 +39,10 @@
#define ZBX_PCP_DERIVED_CONFIG "/etc/zabbix/zbxpcp-derived-metrics.conf"
#endif
+#ifndef ZBX_PCP_CONFIG_VERSION3
+#define ZBX_PCP_CONFIG_VERSION3 "/etc/zabbix/zbxpcp.conf.d/version3"
+#endif
+
/* PCP includes. */
#include "pmapi.h"
#include "impl.h"
@@ -46,12 +50,14 @@
/* Zabbix includes. */
#include "module.h"
+static int zbx_version = 2; /* back-compat */
+
/*
* PCP connection
*/
static int ctx = -1;
-int zbx_module_pcp_connect()
+static int zbx_module_pcp_connect()
{
/* Load possible derived metric definitions. */
if (access(ZBX_PCP_DERIVED_CONFIG, F_OK ) != -1)
@@ -61,7 +67,7 @@ int zbx_module_pcp_connect()
return ctx;
}
-int zbx_module_pcp_disconnect()
+static int zbx_module_pcp_disconnect()
{
return pmDestroyContext(ctx);
}
@@ -73,6 +79,8 @@ int zbx_module_init()
{
if (zbx_module_pcp_connect() < 0)
return ZBX_MODULE_FAIL;
+ if (access(ZBX_PCP_CONFIG_VERSION3, F_OK) != -1)
+ zbx_version = 3;
return ZBX_MODULE_OK;
}
@@ -83,7 +91,7 @@ int zbx_module_api_version()
static int metric_count = 0;
static ZBX_METRIC *metrics = NULL;
-void zbx_module_pcp_add_metric(const char *name);
+static void zbx_module_pcp_add_metric(const char *name);
ZBX_METRIC *zbx_module_item_list()
{
@@ -119,9 +127,10 @@ int zbx_module_uninit()
/*
* Zabbix/PCP connection
*/
-int zbx_module_pcp_fetch_metric(AGENT_REQUEST *request, AGENT_RESULT *result);
+static int zbx_module2_pcp_fetch_metric(AGENT_REQUEST *, AGENT_RESULT_V2 *);
+static int zbx_module3_pcp_fetch_metric(AGENT_REQUEST *, AGENT_RESULT_V3 *);
-void zbx_module_pcp_add_metric(const char *name)
+static void zbx_module_pcp_add_metric(const char *name)
{
int sts;
pmID pmid[1];
@@ -162,12 +171,15 @@ void zbx_module_pcp_add_metric(const char *name)
if (metrics == NULL) { metrics = mptr; free(metric); free(param); return; }
metrics[metric_count].key = metric;
metrics[metric_count].flags = flags;
- metrics[metric_count].function = zbx_module_pcp_fetch_metric;
+ if (zbx_version >= 3)
+ metrics[metric_count].function = zbx_module3_pcp_fetch_metric;
+ else
+ metrics[metric_count].function = zbx_module2_pcp_fetch_metric;
metrics[metric_count].test_param = param;
metric_count++;
}
-int zbx_module_pcp_fetch_metric(AGENT_REQUEST *request, AGENT_RESULT *result)
+static int zbx_module_pcp_fetch_metric(AGENT_REQUEST *request, int *type, pmAtomValue *atom, char **errmsg)
{
int sts;
char *metric[] = { request->key + strlen(ZBX_PCP_METRIC_PREFIX) };
@@ -175,7 +187,6 @@ int zbx_module_pcp_fetch_metric(AGENT_REQUEST *request, AGENT_RESULT *result)
pmID pmid[1];
pmDesc desc[1];
pmResult *rp;
- pmAtomValue atom;
int iid = 0;
int i;
@@ -188,9 +199,8 @@ int zbx_module_pcp_fetch_metric(AGENT_REQUEST *request, AGENT_RESULT *result)
inst = get_rparam(request, 0);
break;
default:
- SET_MSG_RESULT(result, strdup("Extraneous instance specification."));
+ *errmsg = "Extraneous instance specification.";
return SYSINFO_RET_FAIL;
- break;
}
/* Try to reconnect if the initial lookup fails. */
@@ -198,7 +208,7 @@ int zbx_module_pcp_fetch_metric(AGENT_REQUEST *request, AGENT_RESULT *result)
if (sts < 0 && (sts == PM_ERR_IPC || sts == -ECONNRESET)) {
ctx = pmReconnectContext(ctx);
if (ctx < 0) {
- SET_MSG_RESULT(result, strdup("Not connected to pmcd."));
+ *errmsg = "Not connected to pmcd.";
return SYSINFO_RET_FAIL;
}
sts = pmLookupName(1, metric, pmid);
@@ -209,18 +219,18 @@ int zbx_module_pcp_fetch_metric(AGENT_REQUEST *request, AGENT_RESULT *result)
sts = pmLookupDesc(pmid[0], desc);
if (sts < 0) return SYSINFO_RET_FAIL;
if (inst != NULL && desc[0].indom == PM_INDOM_NULL) {
- SET_MSG_RESULT(result, strdup("Extraneous instance specification."));
+ *errmsg = "Extraneous instance specification.";
return SYSINFO_RET_FAIL;
}
if ((inst == NULL && desc[0].indom != PM_INDOM_NULL) ||
(request->nparam == 1 && !strlen(inst))) {
- SET_MSG_RESULT(result, strdup("Missing instance specification."));
+ *errmsg = "Missing instance specification.";
return SYSINFO_RET_FAIL;
}
if (desc[0].indom != PM_INDOM_NULL) {
iid = pmLookupInDom(desc[0].indom, inst);
if (iid < 0) {
- SET_MSG_RESULT(result, strdup("Instance not available."));
+ *errmsg = "Instance not available.";
return SYSINFO_RET_FAIL;
}
}
@@ -230,7 +240,7 @@ int zbx_module_pcp_fetch_metric(AGENT_REQUEST *request, AGENT_RESULT *result)
if (sts < 0) return SYSINFO_RET_FAIL;
if (rp->vset[0]->numval < 1) {
pmFreeResult(rp);
- SET_MSG_RESULT(result, strdup("No value available."));
+ *errmsg = "No value available.";
return SYSINFO_RET_FAIL;
}
@@ -245,12 +255,30 @@ int zbx_module_pcp_fetch_metric(AGENT_REQUEST *request, AGENT_RESULT *result)
/* Extract the wanted value. */
sts = pmExtractValue(rp->vset[0]->valfmt, &rp->vset[0]->vlist[i],
- desc[0].type, &atom, desc[0].type);
+ desc[0].type, atom, desc[0].type);
pmFreeResult(rp);
if (sts < 0) return SYSINFO_RET_FAIL;
+ *type = desc[0].type;
+
+ /* Success. */
+ return SYSINFO_RET_OK;
+}
+
+static int zbx_module3_pcp_fetch_metric(AGENT_REQUEST *request, AGENT_RESULT_V3 *result)
+{
+ char *errmsg = NULL;
+ pmAtomValue atom;
+ int type;
+ int sts;
- /* Hand it to the caller. */
- switch(desc[0].type) {
+ /* note: SET_*_RESULT macros evaluate to different code for v2/v3 */
+ sts = zbx_module_pcp_fetch_metric(request, &type, &atom, &errmsg);
+ if (sts < 0) {
+ if (errmsg)
+ SET_MSG_RESULT(result, strdup(errmsg));
+ return sts;
+ }
+ switch (type) {
case PM_TYPE_32:
SET_UI64_RESULT(result, atom.l);
break;
@@ -274,10 +302,53 @@ int zbx_module_pcp_fetch_metric(AGENT_REQUEST *request, AGENT_RESULT *result)
break;
default:
SET_MSG_RESULT(result, strdup("Unsupported metric value type."));
- return SYSINFO_RET_FAIL;
+ sts = SYSINFO_RET_FAIL;
break;
}
- /* Success. */
- return SYSINFO_RET_OK;
+ return sts;
+}
+
+static int zbx_module2_pcp_fetch_metric(AGENT_REQUEST *request, AGENT_RESULT_V2 *result)
+{
+ char *errmsg = NULL;
+ pmAtomValue atom;
+ int type;
+ int sts;
+
+ /* note: SET_*_RESULT macros evaluate to different code for v2/v3 */
+ sts = zbx_module_pcp_fetch_metric(request, &type, &atom, &errmsg);
+ if (sts < 0) {
+ if (errmsg)
+ SET_MSG_RESULT(result, strdup(errmsg));
+ return sts;
+ }
+ switch (type) {
+ case PM_TYPE_32:
+ SET_UI64_RESULT(result, atom.l);
+ break;
+ case PM_TYPE_U32:
+ SET_UI64_RESULT(result, atom.ul);
+ break;
+ case PM_TYPE_64:
+ SET_UI64_RESULT(result, atom.ll);
+ break;
+ case PM_TYPE_U64:
+ SET_UI64_RESULT(result, atom.ull);
+ break;
+ case PM_TYPE_FLOAT:
+ SET_DBL_RESULT(result, atom.f);
+ break;
+ case PM_TYPE_DOUBLE:
+ SET_DBL_RESULT(result, atom.d);
+ break;
+ case PM_TYPE_STRING:
+ SET_STR_RESULT(result, strdup(atom.cp));
+ break;
+ default:
+ SET_MSG_RESULT(result, strdup("Unsupported metric value type."));
+ sts = SYSINFO_RET_FAIL;
+ break;
+ }
+ return sts;
}
From fche@redhat.com Fri Mar 4 11:20:49 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
by oss.sgi.com (Postfix) with ESMTP id CF33B7CAF
for ; Fri, 4 Mar 2016 11:20:49 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
by relay3.corp.sgi.com (Postfix) with ESMTP id 52CEBAC001
for ; Fri, 4 Mar 2016 09:20:46 -0800 (PST)
X-ASG-Debug-ID: 1457112042-04bdf002ec1ac3e0001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id OwRsmc3EedwR60Ot (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Fri, 04 Mar 2016 09:20:42 -0800 (PST)
X-Barracuda-Envelope-From: fche@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx14.intmail.prod.int.phx2.redhat.com (int-mx14.intmail.prod.int.phx2.redhat.com [10.5.11.27])
by mx1.redhat.com (Postfix) with ESMTPS id 1A4D364D0F
for ; Fri, 4 Mar 2016 17:20:42 +0000 (UTC)
Received: from fche.csb (vpn-58-5.rdu2.redhat.com [10.10.58.5])
by int-mx14.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u24HKfW7010686;
Fri, 4 Mar 2016 12:20:41 -0500
Received: by fche.csb (Postfix, from userid 2569)
id EEB125868D; Fri, 4 Mar 2016 12:20:42 -0500 (EST)
To: Nathan Scott
Cc: Marko Myllynen , pcp@oss.sgi.com
Subject: Re: [PATCH] initial pass at fixing zbxpcp.so for Zabbix v3
References: <1240982544.27234304.1457065841902.JavaMail.zimbra@redhat.com>
<675485289.27235420.1457066754101.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: [PATCH] initial pass at fixing zbxpcp.so for Zabbix v3
From: fche@redhat.com (Frank Ch. Eigler)
Date: Fri, 04 Mar 2016 12:20:42 -0500
In-Reply-To: <675485289.27235420.1457066754101.JavaMail.zimbra@redhat.com> (Nathan Scott's message of "Thu, 3 Mar 2016 23:45:54 -0500 (EST)")
Message-ID:
User-Agent: Gnus/5.1008 (Gnus v5.10.8) Emacs/21.4 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.27
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]); Fri, 04 Mar 2016 17:20:42 +0000 (UTC)
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1457112042
X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384
X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
nathans wrote:
> [...] This switches between using v2/v3 interface based on presence
> or lack of a named file. [...]
This wouldn't be quite so bad if that file name (and others!) weren't
hard-compiled into the zbxpcp.so binary. Instead zbxpcp.so could read
a config file from /etc/pcp with such site-specific installation path
names ... in which case that file could also spell out the v2 vs. v3
situation, and other useful stuff like zbxpcp pmNewContext parameters
(so it's not restricted to "localhost").
- FChE
From kenj@internode.on.net Sat Mar 5 15:26:51 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
by oss.sgi.com (Postfix) with ESMTP id 1904F7F5A
for ; Sat, 5 Mar 2016 15:26:51 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
by relay3.corp.sgi.com (Postfix) with ESMTP id 921E5AC001
for ; Sat, 5 Mar 2016 13:26:47 -0800 (PST)
X-ASG-Debug-ID: 1457213203-04cb6c58301aea50001-S8gJnT
Received: from ipmail06.adl6.internode.on.net (ipmail06.adl6.internode.on.net [150.101.137.145]) by cuda.sgi.com with ESMTP id hw1KgS120KIBx0bw for ; Sat, 05 Mar 2016 13:26:44 -0800 (PST)
X-Barracuda-Envelope-From: kenj@internode.on.net
X-Barracuda-Apparent-Source-IP: 150.101.137.145
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: A2CtCQC7TttWPGcXLHldgzqBP4Jtg3mGSJ1GkUOGCQQCAoEiTQEBAQEBAQcBAQEBQUCEQQEBAQQIAhkFLi8BAwIGAxEEAQEDAiMDAgIZIAoDCQgCBBMLBYgRsESOVyp7iVmHOoE6BZJ6hDABgUybGo5VhFooLolAAQEB
Received: from ppp121-44-23-103.lns20.syd4.internode.on.net (HELO bozohorize) ([121.44.23.103])
by ipmail06.adl6.internode.on.net with ESMTP; 06 Mar 2016 07:56:40 +1030
From: "Ken McDonell"
To: "'PCP'"
References: <56D627B6.6000900@internode.on.net>
In-Reply-To: <56D627B6.6000900@internode.on.net>
Subject: RE: Yet another Python snarfoo
Date: Sun, 6 Mar 2016 08:26:38 +1100
X-ASG-Orig-Subj: RE: Yet another Python snarfoo
Message-ID: <001601d17725$b4f20430$1ed60c90$@internode.on.net>
MIME-Version: 1.0
Content-Type: text/plain;
charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Mailer: Microsoft Outlook 15.0
Thread-Index: AQLvqQLv/fNB2KVOfunHCyODAhCp750O34vQ
Content-Language: en-au
X-Barracuda-Connect: ipmail06.adl6.internode.on.net[150.101.137.145]
X-Barracuda-Start-Time: 1457213204
X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.01
X-Barracuda-Spam-Status: No, SCORE=0.01 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=THREAD_INDEX
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27608
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
I've been beating my head against this brick wall for a few days, and =
would really like some pain relief ...
> -----Original Message-----
> From: Ken McDonell [mailto:kenj@internode.on.net]
> Sent: Wednesday, 2 March 2016 10:37 AM
> To: PCP
> Subject: Yet another Python snarfoo
> ...
> On Mac OS X we've carefully moved PCP artifacts out of /usr to avoid
> installer issues ... this has be necessary on some of the *BSD
> platforms also, so this is not confined to Mac OS X (although that's
> where I'm battling with it at present).
> ...
> As a consequence ...
> ...
> 2. Because pcp(1) is not in /usr/bin on these platforms, none of the
> following will work in a fit:
>=20
> src/pmclient/pmclient_fg.python:#! /usr/bin/pcp python
> ...
The problem is worse than this ... on Mac OS X, shebang (#!) only works =
if the next argument is an executable binary (and not a shell scripts =
like pcp(1)) ... I have days of bloody evidence to support this claim.
When I eventually discovered this I wrote pmpython.c to run =
$PCP_PYTHON_PROG from the env or pcp.conf and changed a couple of the =
Python scripts to use
#!/usr/bin/env pmpython
which demonstrably works.
Anyone see a problem with globally replacing the existing #!/usr/bin/pcp =
python with the above for Python scripts in PCP?
From nscott@redhat.com Sat Mar 5 16:41:58 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
by oss.sgi.com (Postfix) with ESMTP id 4BEF27F54
for ; Sat, 5 Mar 2016 16:41:58 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
by relay3.corp.sgi.com (Postfix) with ESMTP id B8955AC002
for ; Sat, 5 Mar 2016 14:41:54 -0800 (PST)
X-ASG-Debug-ID: 1457217712-04cbb00b711e3880001-S8gJnT
Received: from mx3-phx2.redhat.com (mx3-phx2.redhat.com [209.132.183.24]) by cuda.sgi.com with ESMTP id a7BUF7FEo2mCRBOO (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Sat, 05 Mar 2016 14:41:52 -0800 (PST)
X-Barracuda-Envelope-From: nscott@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.24
Received: from zmail20.collab.prod.int.phx2.redhat.com (zmail20.collab.prod.int.phx2.redhat.com [10.5.83.23])
by mx3-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id u25MfmGO020663;
Sat, 5 Mar 2016 17:41:48 -0500
Date: Sat, 5 Mar 2016 17:41:48 -0500 (EST)
From: Nathan Scott
Reply-To: Nathan Scott
To: Ken McDonell
Cc: PCP
Message-ID: <671813428.28024681.1457217708502.JavaMail.zimbra@redhat.com>
In-Reply-To: <001601d17725$b4f20430$1ed60c90$@internode.on.net>
References: <56D627B6.6000900@internode.on.net> <001601d17725$b4f20430$1ed60c90$@internode.on.net>
Subject: Re: [pcp] Yet another Python snarfoo
MIME-Version: 1.0
X-ASG-Orig-Subj: Re: [pcp] Yet another Python snarfoo
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Originating-IP: [10.64.48.220]
X-Mailer: Zimbra 8.0.6_GA_5922 (ZimbraWebClient - FF44 (Linux)/8.0.6_GA_5922)
Thread-Topic: Yet another Python snarfoo
Thread-Index: AQLvqQLv/fNB2KVOfunHCyODAhCp750O34vQfNdbKPM=
X-Barracuda-Connect: mx3-phx2.redhat.com[209.132.183.24]
X-Barracuda-Start-Time: 1457217712
X-Barracuda-Encrypted: DHE-RSA-AES256-SHA
X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.02
X-Barracuda-Spam-Status: No, SCORE=0.02 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, THREAD_INDEX, THREAD_TOPIC
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27608
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
0.01 THREAD_INDEX thread-index: AcO7Y8iR61tzADqsRmmc5wNiFHEOig==
0.01 THREAD_TOPIC Thread-Topic: ...(Japanese Subject)...
0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header
Hi Ken,
----- Original Message -----
> I've been beating my head against this brick wall for a few days, and would
> really like some pain relief ...
>
:) Thanks for fighting the good fight!
> [...]
> #!/usr/bin/env pmpython
>
> which demonstrably works.
>
> Anyone see a problem with globally replacing the existing #!/usr/bin/pcp
> python with the above for Python scripts in PCP?
>
Seems a good approach to me (we can drop pcp-python then? or make it a symlink
I guess, for back-compat with existing scripts in the wild).
cheers.
--
Nathan
From sitaramshelke@gmail.com Sun Mar 6 03:20:11 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level: **
X-Spam-Status: No, score=2.1 required=5.0 tests=FREEMAIL_FROM,
HTML_IMAGE_ONLY_20,HTML_MESSAGE,T_DKIM_INVALID,T_REMOTE_IMAGE autolearn=no
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
by oss.sgi.com (Postfix) with ESMTP id 964D77F5E
for ; Sun, 6 Mar 2016 03:20:11 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
by relay3.corp.sgi.com (Postfix) with ESMTP id EF07CAC001
for ; Sun, 6 Mar 2016 01:20:07 -0800 (PST)
X-ASG-Debug-ID: 1457256002-04cbb00b701eeac0001-S8gJnT
Received: from mail-vk0-f51.google.com (mail-vk0-f51.google.com [209.85.213.51]) by cuda.sgi.com with ESMTP id l6mWaCKWpyeQDILi (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 06 Mar 2016 01:20:02 -0800 (PST)
X-Barracuda-Envelope-From: sitaramshelke@gmail.com
X-Barracuda-Apparent-Source-IP: 209.85.213.51
Received: by mail-vk0-f51.google.com with SMTP id c3so91566001vkb.3
for ; Sun, 06 Mar 2016 01:20:02 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20120113;
h=mime-version:date:message-id:subject:from:to;
bh=m/FXaZnaeAkiJYZDUzETVXVP9cBKKgqPzME3j93opsY=;
b=JG2zXtI77MmX0pMSk7CJK785R0gvirKVsN/f65NbWhejdV+bk0xaVuT57jTCRb1Ch+
J8HjZXXUno1Y1INyZuJdwW1g2i4iRhDit/uSKZLhy6ClCWttwGE+LN7vegZbTIbdeUxb
r2A7FOe5fd5uZJOcj0Uobro9VmhJ11dcnBlBUd6X1N4g8/x8+Tyf2jqX8agXnF1J3ZRV
hagqHJfsh7Be4u2fAWJ6RVG7IF7AumU039y1OSE1ZVF6kO/8PryzLUmCxur1fKPbGVBF
IN88iCZPoOdiuTRhkMAJVqBtf0TtguCrz5fnG8g7U+0w9H8sxwcPx7x1iw9ZcuHud/2Y
eeTg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20130820;
h=x-gm-message-state:mime-version:date:message-id:subject:from:to;
bh=m/FXaZnaeAkiJYZDUzETVXVP9cBKKgqPzME3j93opsY=;
b=gfl7xaeblJE7Yr8JdW+ln4ZYlBOSdhpsmjKMP08yjP1ebOlA5GSD/59zR09pp+eh2D
Ybq5cmhEe5q9qpMwRlWBnZH94vaLmTbWnx+/NyA8y9ZS6avm+gdr17PtlF/hV9JzDhr3
HVQvYEAqEutV6hFdNyjdQ4wz9v0yQitmAuO2LpymKPPB7NXoBrM/4WgLmIeQTRr/AaMi
FQFknGWXCDn7boo6x4vDIEiF6pn4zkb/tQ5WRBFVEtyPrOK29OQxsR6W/psFq4HewRGi
C9TSEDZQR3QDkjEgRkx5/dPr04TtJkPg9zvCK9mxX4dK/XVgbg8WW0vzQJ173aB2YeP9
FXkQ==
X-Gm-Message-State: AD7BkJLvx9IvPDn4d6u/UQnxR0j9U8b9GHcJnDj0VpwbnLoUdhHrYXs3EoGWbzoqoKdPbVFR7Spn6lhZYCT2eQ==
MIME-Version: 1.0
X-Received: by 10.31.52.195 with SMTP id b186mr13660402vka.39.1457256001823;
Sun, 06 Mar 2016 01:20:01 -0800 (PST)
Received: by 10.159.41.36 with HTTP; Sun, 6 Mar 2016 01:20:01 -0800 (PST)
Date: Sun, 6 Mar 2016 14:50:01 +0530
Message-ID:
Subject: Interested to contribute to PCP in GSoC 16
From: Sitaram Shelke
X-ASG-Orig-Subj: Interested to contribute to PCP in GSoC 16
To: pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=001a1143e54255ecea052d5dda7d
X-Barracuda-Connect: mail-vk0-f51.google.com[209.85.213.51]
X-Barracuda-Start-Time: 1457256002
X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256
X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-BRTS-Evidence: sitaramshelke.github.io
X-Barracuda-Spam-Score: 0.70
X-Barracuda-Spam-Status: No, SCORE=0.70 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=DKIM_SIGNED, DKIM_VERIFIED, HTML_IMAGE_ONLY_20, HTML_IMAGE_ONLY_20_2, HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27612
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
-0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes
verification
0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature
0.00 HTML_IMAGE_ONLY_20 BODY: HTML: images with 1600-2000 bytes of words
0.00 HTML_MESSAGE BODY: HTML included in message
0.70 HTML_IMAGE_ONLY_20_2 HTML: images with 1600-2000 bytes of words
--001a1143e54255ecea052d5dda7d
Content-Type: text/plain; charset=UTF-8
Hello Community,
I am Sitaram, and I am interested to contribute to Performance Co-Pilot. I
read the the ideas listed on the GSoC Ideas page and found these two ideas
interesting.
1. Performance metric extraction using Go language agents
2. PCP versions of console tools
I read the reply to all of the previous discussions on this mailing list
and read the programmers guide for pcp mentioned in one of the reply.
I understood the basic concepts used in pcp like pcp daemon, domain agents,
collectors and metrics and I have built pcp from source using the
instructions provided in the source ( Thanks to Nathan for helping with the
errors) and tried few commands to test their working as explained on the
pcp website. I also ran some QA tests (not the complete test suite though).
Lukas has explained about first task in one of the replies earlier and I
would like to know more about the second task.
About me:
I am an undergraduate student at Pune Institute of Computer Technology.
I have worked with golang and python as well as postgresql and mysql.
I have some experience in Devops, especially AWS technologies as well as CI
tools like Jenkins
I have worked as an intern where my responsibility was the backend server
which written in golang, and deployed on AWS
I blog at sitaramshelke.github.io and my Freenode and Telegram handle is
'sitaramshelke'.
--
-sitaram
--001a1143e54255ecea052d5dda7d
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Hello Community,
I am Sitaram, and I am interested to c=
ontribute to=20
Performance Co-Pilot. I read the the ideas listed on the GSoC Ideas page
and found these two ideas interesting.
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0 1. Performance metric extraction using Go language agents
=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 2. PCP versions of console tools=
I
read the reply to all of the previous discussions on this mailing list=20
and read the programmers guide for pcp mentioned in one of the reply.
I=
understood the=20
basic concepts used in pcp like pcp daemon, domain agents, collectors and
metrics and I have built pcp from source using the instructions=20
provided in the source ( Thanks to Nathan for helping with the errors)=20
and tried few commands to test their working as explained on the pcp=20
website. I also ran some QA tests (not the complete test suite though).
=
Lukas has explained about first task in one of the replies earlier and I wo=
uld like to know more about the second task.
About me:
=C2=A0=C2=
=A0 I am an undergraduate student at Pune Institute of Computer Technology.=
I have worked with=C2=A0 golang and python as well as postgresql and mysql=
.
I have some experience in Devops, especially AWS technologies as well =
as CI tools like Jenkins
I have worked as an intern where my responsibil=
ity was the backend server which written in golang, and deployed on AWS
=
I blog at
sitaramshelke.github.i=
o and my Freenode and Telegram handle is 'sitaramshelke'.
=
--
-sitaram
--001a1143e54255ecea052d5dda7d--
From charles.cui1984@gmail.com Sun Mar 6 12:27:29 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.3 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,
FREEMAIL_FROM,HTML_MESSAGE,T_DKIM_INVALID autolearn=no version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
by oss.sgi.com (Postfix) with ESMTP id 0B8F57F63
for ; Sun, 6 Mar 2016 12:27:29 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
by relay1.corp.sgi.com (Postfix) with ESMTP id BDAB98F8033
for ; Sun, 6 Mar 2016 10:27:22 -0800 (PST)
X-ASG-Debug-ID: 1457288835-04bdf002ec1f3cf0001-S8gJnT
Received: from mail-lb0-f174.google.com (mail-lb0-f174.google.com [209.85.217.174]) by cuda.sgi.com with ESMTP id UB2v08iCf4YvfqZp (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Sun, 06 Mar 2016 10:27:16 -0800 (PST)
X-Barracuda-Envelope-From: charles.cui1984@gmail.com
X-Barracuda-Apparent-Source-IP: 209.85.217.174
Received: by mail-lb0-f174.google.com with SMTP id k15so108805036lbg.0
for ; Sun, 06 Mar 2016 10:27:16 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20120113;
h=mime-version:in-reply-to:references:date:message-id:subject:from:to
:cc;
bh=9a+Vf751+DJm2EZBIfkt8GkxL0ayrDSeR+5T0uGbsxw=;
b=kMZvUBJv9LzSY03g7Sm8wG+i0lh0zJgwuCDwi0IP45rePRlvDJLOrD2tfTdoR4q/ts
AVEHkxW56gfjJwI+hQ9mdnWJ9TkHM5jczujl1IeeCJIylBG+BRgv54urEB2zJOt4yeCc
xWPzbKuxKJTdJEdYryNT+F4sxTv+8vYnnxYZffmPnaILnoyqmmMYHZtsE3zsBGhfdiK7
am+jFgZgEhN4VsMmLNX+A4qsSFC5Qm39GCE71Ux58ETyswvTqH4i7OYJ0p42dX2tUQ9e
ZJQurhtuSll1naoU0fyuQ0BSq1kHfkaFBGhRThlN+cyl+jZKbyMqQrbfA2Woi1X/RiEN
DTeg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20130820;
h=x-gm-message-state:mime-version:in-reply-to:references:date
:message-id:subject:from:to:cc;
bh=9a+Vf751+DJm2EZBIfkt8GkxL0ayrDSeR+5T0uGbsxw=;
b=TJKrbGzFupWQyJ5R1K40hd+7BHSFsxWMN/W029ePG61XXRuuC6OCshfVNFu5no3rYG
7ee/v+Lj4qKvYDdKd4XLhv/BhUDifbWG0dH0Jd7vOhbs/yWbkc5Lxq1aEpARNOQTJ2go
K0c6BqkcmPw9CUDjKJVY30wkMigqzy0j/fOJ/5mNgs4vljDfXxXkhgt5YEzVSjIWwEhc
cMzS80cpJ3v2xeUm+CUysYVmhZvtxa0mpaTK1H+z0U2il7DB0puXDortow9I8cTH5kVZ
+vUBOgViupA2rhbA+IFann8m8y+8OjKjccPdTKuSRngAH01TzLgJU3KKBKflKqovsMRC
73MQ==
X-Gm-Message-State: AD7BkJIE5AO/qODk5tjWjHqTVxrWPJVWRxfRjjZD2/1tLx9/DH5q+Z1QS+gycnBH0NJ/dYzkPQHTaLmeUJvf0A==
MIME-Version: 1.0
X-Received: by 10.25.170.85 with SMTP id t82mr4885880lfe.57.1457288834761;
Sun, 06 Mar 2016 10:27:14 -0800 (PST)
Received: by 10.25.86.68 with HTTP; Sun, 6 Mar 2016 10:27:14 -0800 (PST)
In-Reply-To:
References:
<87si089svf.fsf@redhat.com>
<874mcntvhv.fsf@redhat.com>
Date: Sun, 6 Mar 2016 10:27:14 -0800
Message-ID:
Subject: Re: [pcp] GSOC 2016 idea for PCP
From: Charles Cui
X-ASG-Orig-Subj: Re: [pcp] GSOC 2016 idea for PCP
To: Lukas Berk
Cc: pcp@oss.sgi.com
Content-Type: multipart/alternative; boundary=001a114113e854ce1b052d657f2a
X-Barracuda-Connect: mail-lb0-f174.google.com[209.85.217.174]
X-Barracuda-Start-Time: 1457288836
X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256
X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, DKIM_SIGNED, DKIM_VERIFIED, HTML_MESSAGE
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27621
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header
-0.00 DKIM_VERIFIED Domain Keys Identified Mail: signature passes
verification
0.00 DKIM_SIGNED Domain Keys Identified Mail: message has a signature
0.00 HTML_MESSAGE BODY: HTML included in message
--001a114113e854ce1b052d657f2a
Content-Type: text/plain; charset=UTF-8
Hi Lukas Berk,
Following your advices, I build PCP from source on an Ubuntu system, and
went through a part of the document. PCP project provides a great document,
which answers all of my questions in the build. The whole process is
smooth. However, I have one specific question about the task: I found that
PCP agents (pmdas) are domain related. So, basically you have different
agents for different parts of the system. You can have agent for the kernel
or agent for the database. My question is which agent(s) should I focus
during the project? Do you have specific targets? I plan to look at the
current implementation and prepare for possible challenges.
Thanks, Charles.
2016-03-03 10:18 GMT-08:00 Charles Cui :
> 2016-03-03 7:46 GMT-08:00 Lukas Berk :
>
>> Hi Charles,
>>
>> Charles Cui writes:
>> [...]
>> > Thanks for your reply! To answer your question. Yes, I have been
>> > selected in GSoC 2014
>> > and successfully complete the project. Also thanks for providing such
>> > a great document.
>>
>> Nice! Was that the Plan9 project?
>>
>> > I will read it carefully. By the way, is it enough to familiar with
>> > the documents for the current phase?
>> > Let me know if you want some initial work to be done for this period.
>>
>> I would also recommend trying to build and install pcp from source[1] to
>> start, and perhaps even running the testsuite. What OS do you typically
>> use? (if possible, I would suggest starting with a popular linux distro,
>> it's a bit more simple to setup the build dependencies).
>>
>> Check out the INSTALL file, specifically the 'B. Building From Source'
>> section, where I'd recommend the Makepkgs approach. Please feel free to
>> let me know if you run into any issues at all!
>>
> Great! I will start to run PCP locally on ubuntu. I am sure there will be
> some issues for this project from me. I will let the community know.
>
>>
>> Cheers,
>>
>> Lukas
>>
>> [1] - git://pcp.io/pcp/pcp.git
>>
>
>
--001a114113e854ce1b052d657f2a
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Hi Lukas Berk,
=C2=A0 =C2=A0Following y=
our advices, I build PCP from source on an Ubuntu system, and went through =
a part of the document. PCP project provides a great document, which answer=
s all of my questions in the build. The whole process is smooth. However, I=
have one specific question about the task: I found that PCP agents (pmdas)=
are domain related. So, basically you have different agents for different =
parts of the system. You can have agent for the kernel or agent for the dat=
abase. My question is which agent(s) should I focus during the project? Do =
you have specific targets? I plan to look at the current implementation and=
prepare for possible challenges.=C2=A0
Thanks, Ch=
arles. =C2=A0
--001a114113e854ce1b052d657f2a--
From kenj@internode.on.net Sun Mar 6 19:45:53 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay2.corp.sgi.com [137.38.102.29])
by oss.sgi.com (Postfix) with ESMTP id 6F0687F6A
for ; Sun, 6 Mar 2016 19:45:53 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
by relay2.corp.sgi.com (Postfix) with ESMTP id 4287030404E
for ; Sun, 6 Mar 2016 17:45:52 -0800 (PST)
X-ASG-Debug-ID: 1457315143-04cbb00b70202d00001-S8gJnT
Received: from ipmail06.adl6.internode.on.net (ipmail06.adl6.internode.on.net [150.101.137.145]) by cuda.sgi.com with ESMTP id 6SHEMEjimLl5l0qC for ; Sun, 06 Mar 2016 17:45:44 -0800 (PST)
X-Barracuda-Envelope-From: kenj@internode.on.net
X-Barracuda-Apparent-Source-IP: 150.101.137.145
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: A2DkAQCl3NxWPBrS0XYNUIQMbYZmtVAZh2MBAQEBAQEHAQEBAUGFKwRRMAYCBRYLAgsDAgECATEnBgIBAbgZZ456e5ETgToFlyqFY4o4jEyOVYRaVok9AQEB
Received: from ppp118-209-210-26.lns20.mel8.internode.on.net (HELO [192.168.1.100]) ([118.209.210.26])
by ipmail06.adl6.internode.on.net with ESMTP; 07 Mar 2016 12:15:43 +1030
To: pcp@oss.sgi.com
Subject: pcp updates: pmpython and mac os x qa
From: Ken McDonell
X-ASG-Orig-Subj: pcp updates: pmpython and mac os x qa
Message-ID: <56DCDD45.9000307@internode.on.net>
Date: Mon, 7 Mar 2016 12:45:41 +1100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
Thunderbird/38.5.1
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Barracuda-Connect: ipmail06.adl6.internode.on.net[150.101.137.145]
X-Barracuda-Start-Time: 1457315144
X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27631
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
Changes committed to git://git.pcp.io/kenj/pcp master
Ken McDonell (8):
pmpython: (new) replacement for "pcp python"
qa/863: (new) exercise pmpython
python scripts and qa: use pmpython
qa/admin/pcp-daily: small change to post QA output filtering
qa/948: filter PMIDs that are only correct for Linux
qa/842: avoid seq(1) and echo -n
qa/929: filter out privateTMP -> TMP
qa/781: filter privateLOGDIR -> LOGDIR
man/man1/pcp-python.1 | 7 +
man/man1/pmpython.1 | 104 ++++++++++++++++++++++++
qa/1081 | 4
qa/781 | 1
qa/842 | 8 +
qa/847 | 4
qa/863 | 37 ++++++++
qa/863.out | 5 +
qa/929 | 1
qa/948 | 2
qa/948.out | 16 +--
qa/admin/pcp-daily | 5 -
qa/group | 1
qa/pmdas/memory_python/pmdamemory_python.python | 2
qa/pmdas/slow_python/pmdaslow_python.python | 2
qa/src/check_import.python | 2
qa/src/fsstats.python | 2
qa/src/pmapi_exceptions.python | 2
qa/src/procpid.python | 2
qa/src/test_mmv.python | 2
qa/src/test_pcp.python | 2
qa/src/test_pcp_getopts.python | 2
qa/src/test_pcp_options.python | 2
qa/src/test_pcp_time.python | 2
qa/src/test_pmcc.python | 2
qa/src/test_pmfg.python | 2
qa/src/test_pmi.python | 2
qa/src/test_pmnswalk.python | 2
qa/src/test_set_source.python | 2
qa/src/test_webapi.python | 2
qa/src/test_webcontainers.python | 2
qa/src/test_webprocesses.python | 2
src/GNUmakefile | 1
src/pcp/collectl/pcp-collectl.py | 2
src/pcp/dmcache/pcp-dmcache.py | 2
src/pcp/free/pcp-free.py | 2
src/pcp/iostat/pcp-iostat.py | 2
src/pcp/numastat/pcp-numastat.py | 2
src/pcp/uptime/pcp-uptime.py | 2
src/pcp/verify/pcp-verify.py | 2
src/pcp2graphite/pcp2graphite.py | 2
src/pmclient/pmclient_fg.python | 2
src/pmdas/gluster/pmdagluster.python | 2
src/pmdas/json/pmdajson.python | 2
src/pmdas/simple/pmdasimple.python | 2
src/pmdas/unbound/GNUmakefile | 1
src/pmdas/unbound/pmdaunbound.python | 2
src/pmdas/zswap/pmdazswap.python | 2
src/pmgadgets/pmgsys.py | 2
src/pmpython/.gitignore | 1
src/pmpython/GNUmakefile | 36 ++++++++
src/pmpython/pmpython.c | 57 +++++++++++++
src/pmrep/pmrep.py | 2
53 files changed, 309 insertions(+), 52 deletions(-)
Details ...
commit 2874635592d31f8e4e15544985f000abe3844e68
Author: Ken McDonell
Date: Mon Mar 7 07:33:12 2016 +1100
qa/781: filter privateLOGDIR -> LOGDIR
On Mac OS X /tmp is a symlink to /private/tmp which can lead to
some pathnames having a /private prefix we don't otherwise expect.
commit 9a5bf4c3a35992db4b777aac6a0f86bb0850c122
Author: Ken McDonell
Date: Mon Mar 7 07:27:54 2016 +1100
qa/929: filter out privateTMP -> TMP
On Mac OS X /tmp is a symlink to /private/tmp which can lead to
some pathnames having a /private prefix we don't otherwise expect.
commit 987a7f819612c5e773b4ff5d34290a31cf0fa210
Author: Ken McDonell
Date: Mon Mar 7 07:22:45 2016 +1100
qa/842: avoid seq(1) and echo -n
Both are non-portable and not necessary.
commit e4edd7b065c4a4b3201e792958915dc3a8d8165c
Author: Ken McDonell
Date: Mon Mar 7 07:18:36 2016 +1100
qa/948: filter PMIDs that are only correct for Linux
commit a4fe3a2ca396ed7c3ba06fbef4323e2046cb4282
Author: Ken McDonell
Date: Mon Mar 7 07:16:22 2016 +1100
qa/admin/pcp-daily: small change to post QA output filtering
commit 35e73f97c66e5916d3d5fc3297c6a65ac910ef7c
Author: Ken McDonell
Date: Sun Mar 6 20:52:14 2016 +1100
python scripts and qa: use pmpython
commit 569977b5bcd6bf91115bdd4130af4704210aad8e
Author: Ken McDonell
Date: Sun Mar 6 20:51:29 2016 +1100
qa/863: (new) exercise pmpython
commit 31132093e543fe7d47321c27d3869047eddcaa1f
Author: Ken McDonell
Date: Sun Mar 6 20:50:12 2016 +1100
pmpython: (new) replacement for "pcp python"
From mgoodwin@redhat.com Sun Mar 6 22:43:56 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
by oss.sgi.com (Postfix) with ESMTP id 271F729E08
for ; Sun, 6 Mar 2016 22:43:56 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
by relay1.corp.sgi.com (Postfix) with ESMTP id EE2CC8F8035
for ; Sun, 6 Mar 2016 20:43:55 -0800 (PST)
X-ASG-Debug-ID: 1457325831-04cb6c58301d2c40001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id cwg1RPEymXcrgNEQ (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Sun, 06 Mar 2016 20:43:51 -0800 (PST)
X-Barracuda-Envelope-From: mgoodwin@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26])
by mx1.redhat.com (Postfix) with ESMTPS id 26E7963175
for ; Mon, 7 Mar 2016 04:43:51 +0000 (UTC)
Received: from [10.64.49.14] (vpn1-49-14.bne.redhat.com [10.64.49.14])
by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u274hnet028933
for ; Sun, 6 Mar 2016 23:43:50 -0500
From: Mark Goodwin
Subject: patch/RFC - install global derived metrics dir and configs
To: pcp
X-ASG-Orig-Subj: patch/RFC - install global derived metrics dir and configs
Message-ID: <56DD0704.3000800@redhat.com>
Date: Mon, 7 Mar 2016 15:43:48 +1100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
Thunderbird/38.6.0
MIME-Version: 1.0
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Mon, 07 Mar 2016 04:43:51 +0000 (UTC)
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1457325831
X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384
X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
RFC - ship a global derived metrics config directory in the pcp package.
$PCP_VAR_DIR/config/derived along with a config for IOSTAT metrics. Other
configs can be added. If this is deemed reasonable, then my next commit
makes pmchart load all configs in this directory by default (and other
generic monitoring tools such as pmrep, pmval, pmdumptext etc could do
the same).
REALLY handy being able to plot iostat metrics in pmchart using these new
configs - especially since they work for existing archives, i.e. no server
side changes needed. This can already be done by setting PCP_DERIVED_CONFIG
in the environment, but try explaining that to new users ...
TODO: deb packaging. By default this ships in the 'pcp' package for RPM
based distros.
Changes committed to git://git.pcp.io/mgoodwin/pcp master
commit 689c031ca6f5db455cce6aab8f145eb5b8029223
Author: Mark Goodwin
Date: Mon Mar 7 15:23:56 2016 +1100
config: install a global derived metrics config directory with iostat.conf
Installs a global derived metrics config dir $PCP_VAR_DIR/config/derived
along with iostat.conf (containing iostat derived metrics definitions).
modified: src/GNUmakefile
new file: src/derived/GNUmakefile
new file: src/derived/iostat.conf
From myllynen@redhat.com Mon Mar 7 04:08:32 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
by oss.sgi.com (Postfix) with ESMTP id F1B067CA1
for ; Mon, 7 Mar 2016 04:08:32 -0600 (CST)
Received: from cuda.sgi.com (cuda1.sgi.com [192.48.157.11])
by relay1.corp.sgi.com (Postfix) with ESMTP id A73448F8035
for ; Mon, 7 Mar 2016 02:08:29 -0800 (PST)
X-ASG-Debug-ID: 1457345301-04bdf002ec211770001-S8gJnT
Received: from mail-wm0-f49.google.com (mail-wm0-f49.google.com [74.125.82.49]) by cuda.sgi.com with ESMTP id 4vInM6HxmCVE4gPq (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Mon, 07 Mar 2016 02:08:22 -0800 (PST)
X-Barracuda-Envelope-From: myllynen@redhat.com
X-Barracuda-Apparent-Source-IP: 74.125.82.49
Received: by mail-wm0-f49.google.com with SMTP id p65so101132109wmp.1
for ; Mon, 07 Mar 2016 02:08:22 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20130820;
h=x-gm-message-state:reply-to:subject:references:to:cc:from
:organization:message-id:date:user-agent:mime-version:in-reply-to
:content-transfer-encoding;
bh=ylMw5Q9P9eAJaSSvzgb/2zAqTuLqNqB6Ju3yfLDYmco=;
b=NOT611jQCxQz2BT8UaZWa5VtpxOYYUdXEUsjVen/looLS1LOwdvUyrAP/hbfyV5Ozm
D5DMR0SKu8oB9lSQb7VgoDB71pXDcgGd2roiv3k08ZVDJPAQ4YNGDnN4+0iUPH5B4YVd
7k+JG42WgtLXEC/qPShFO6MPddInHvb7g9osykoAWPPhNGosUaEtWX7WulFc/hzCAUqf
8/zESMrhA0vOv0RLdLNR632fIUer1krxll7ChtY1rf/47qmu1VJXZScgtSOab+QwvsIY
3KdqaZFVn4nBvISQvrpitQwBXozM9xJb8irGGN6Ru4X6CaiOYa8CUryKt0GI6LVY/zsw
Y1mg==
X-Gm-Message-State: AD7BkJIbAAn7k7otzg4Cd6zC72Sy7WAANmbs2P0WipZnLAi+mmJqa1dyg3yotzJr0XmPRx0/
X-Received: by 10.194.189.231 with SMTP id gl7mr21564717wjc.162.1457345301332;
Mon, 07 Mar 2016 02:08:21 -0800 (PST)
Received: from ?IPv6:2001:14bb:100:24e4:402:1fcc:a508:2? (dyfycl5yfyks85lkkyyby-4.rev.dnainternet.fi. [2001:14bb:100:24e4:402:1fcc:a508:2])
by smtp.gmail.com with ESMTPSA id v1sm17114681wjf.49.2016.03.07.02.08.19
(version=TLSv1/SSLv3 cipher=OTHER);
Mon, 07 Mar 2016 02:08:20 -0800 (PST)
Reply-To: Marko Myllynen
Subject: Re: [PATCH] initial pass at fixing zbxpcp.so for Zabbix v3
References: <675485289.27235420.1457066754101.JavaMail.zimbra@redhat.com>
X-ASG-Orig-Subj: Re: [PATCH] initial pass at fixing zbxpcp.so for Zabbix v3
To: Nathan Scott
Cc: pcp developers
From: Marko Myllynen
Organization: Red Hat
Message-ID: <56DD5312.2030004@redhat.com>
Date: Mon, 7 Mar 2016 12:08:18 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
Thunderbird/38.6.0
MIME-Version: 1.0
In-Reply-To: <675485289.27235420.1457066754101.JavaMail.zimbra@redhat.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Barracuda-Connect: mail-wm0-f49.google.com[74.125.82.49]
X-Barracuda-Start-Time: 1457345302
X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256
X-Barracuda-URL: https://192.48.157.11:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.01
X-Barracuda-Spam-Status: No, SCORE=0.01 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=BSF_SC0_MISMATCH_TO, BSF_SC0_SA_TO_FROM_DOMAIN_MATCH
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27640
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
0.00 BSF_SC0_MISMATCH_TO Envelope rcpt doesn't match header
0.01 BSF_SC0_SA_TO_FROM_DOMAIN_MATCH Sender Domain Matches Recipient
Domain
Hi Nathan,
On 2016-03-04 06:45, Nathan Scott wrote:
>
> This should fix the problematic interactions between zbxpcp and the
> new version of Zabbix with the ABI/API break. I noticed the problem
> structure (AGENT_RESULT) can actually be determined at runtime, so
> we can (at least in theory) solve this via a single .so as we first
> hoped. We don't ever use zbx_log_t (which also changed), and it is
> only ever referenced via pointer, so I made that opaque and dropped
> its definition entirely.
>
> This switches between using v2/v3 interface based on presence or lack
> of a named file. This is a bit suboptimal as it still involves some
> manual user interaction, but its better than having multiple shared
> libraries I think. If we could find some reliable way of detecting a
> Zabbix 3 vs 2 install (?) at runtime, we'd be laughing. Something in
> the environment, or a file we could test for? A symbol we could look
> for with dlsym, even? Anyway, this will probably do for now.
Thanks, this is a nice idea and the patch in general looks but I think I
agree with Frank that it'd be better to have a configuration file than a
.d directory. However, not sure would it have to be under /etc/pcp or
would /etc/zabbix be ok as well. Initially perhaps only the Zabbix
version and PCP source could be supported there. Do you think you could
update the patch in that regard, I can do some testing and also update
the man page accordingly?
> I don't know whether we need to set the new AGENT_RESULT fields, i.e.
> lastlogsize and mtime, in the zbxpcp.so v3 callback code - any idea?
I don't think so, the macros don't do it and a quick look at the agent
side doesn't reveal anything either.
Thanks,
--
Marko Myllynen
From myllynen@redhat.com Mon Mar 7 07:39:51 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
by oss.sgi.com (Postfix) with ESMTP id 679347CA0
for ; Mon, 7 Mar 2016 07:39:51 -0600 (CST)
Received: from cuda.sgi.com (cuda3.sgi.com [192.48.176.15])
by relay1.corp.sgi.com (Postfix) with ESMTP id 332EA8F8035
for ; Mon, 7 Mar 2016 05:39:51 -0800 (PST)
X-ASG-Debug-ID: 1457357988-04cb6c58331e9fc0001-S8gJnT
Received: from mail-wm0-f54.google.com (mail-wm0-f54.google.com [74.125.82.54]) by cuda.sgi.com with ESMTP id 0uOa8pLJ4i4l6aca (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Mon, 07 Mar 2016 05:39:48 -0800 (PST)
X-Barracuda-Envelope-From: myllynen@redhat.com
X-Barracuda-Apparent-Source-IP: 74.125.82.54
Received: by mail-wm0-f54.google.com with SMTP id l68so86471862wml.0
for ; Mon, 07 Mar 2016 05:39:48 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20130820;
h=x-gm-message-state:reply-to:to:from:subject:organization:message-id
:date:user-agent:mime-version:content-transfer-encoding;
bh=5xOwaNyTJnCZBHAdsHf8mykFkPDkGnZnwbZTXGqROUE=;
b=QZOg/3hpVOoxzKC0yJE8qK3vdtaO7LAm/jS05jTnFhVlkW0k26fA2PdDtZGGITpPHm
bupoMeCD5cq1IbgZYAfqtBccJ8TbWK/GuovdNzkgF7i7VDHZrH7go3E3wjFhxYAWmiDY
Nj+Izdg+elgFLuWV8f+NJjPgzAaFGIOQ8BbIcJ1LjTEFl5AzGaZOpywagUoj1WSmtc6x
1ZVoXL2Okv1INTciU/TlpAjNrwQNsDMQvgyOpRq1Rn9eTpYgBKhajqcMzUbvxUHBLyYl
USUKIZ29OoBYtEh7slnfrkY9T6xQ3QignBCN7wCeGZjnP00sJPWOhyZikuq6f6C3xuJw
J8Mg==
X-Gm-Message-State: AD7BkJLcJy7KgxyRyK5PKvZzGuDLc5QRfUrYDpWX97C6UmAn6YJK4SWmnKS79yv5FfL7iFI7
X-Received: by 10.194.216.40 with SMTP id on8mr25491355wjc.40.1457357987833;
Mon, 07 Mar 2016 05:39:47 -0800 (PST)
Received: from ?IPv6:2001:14bb:100:24e4:402:1fcc:a508:2? (dyfycl5yfyks85lkkyyby-4.rev.dnainternet.fi. [2001:14bb:100:24e4:402:1fcc:a508:2])
by smtp.gmail.com with ESMTPSA id hm9sm18054582wjb.34.2016.03.07.05.39.46
for
(version=TLSv1/SSLv3 cipher=OTHER);
Mon, 07 Mar 2016 05:39:47 -0800 (PST)
Reply-To: Marko Myllynen
To: pcp developers
From: Marko Myllynen
Subject: Script to run individual QA test cases
Organization: Red Hat
X-ASG-Orig-Subj: Script to run individual QA test cases
Message-ID: <56DD84A1.8020000@redhat.com>
Date: Mon, 7 Mar 2016 15:39:45 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
Thunderbird/38.6.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Barracuda-Connect: mail-wm0-f54.google.com[74.125.82.54]
X-Barracuda-Start-Time: 1457357988
X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256
X-Barracuda-URL: https://192.48.176.15:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27643
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
Hi,
FWIW, I'm using the below kind of script to setup my environment
to allow me running individual QA test cases with a non-system
PCP installation. This is handy when working on some completely
isolated features.
1) PCP installation under /tmp/pcp:
SYSTEMD_SYSTEMUNITDIR=/tmp/pcp/lib/systemd/system ./configure --prefix=/tmp/pcp --with-secure-sockets=no --with-static-probes=yes --with-infiniband=no --with-discovery=no --with-webapi=no --with-manager=no --with-pmdajson=no --with-perfevent=yes --with-qt=yes --with-x=yes
make -j 4
make -j 4 install
2) Setup the test environment and run a test case to make sure
all is ok:
#!/bin/sh
PCP_INSTALL=/tmp/pcp
PCP_QA_TEST=$PCP_INSTALL/var/lib/pcp/testsuite
PCP_PYTHON_VER=3.3
mkdir -p $PCP_INSTALL/var/tmp $PCP_INSTALL/var/lib/pcp/pmns
cp -p /NotBackedUp/src/pcp.build/src/pmns/* /tmp/pcp/var/lib/pcp/pmns/
printf "#!/bin/sh\nexit 0\n" > $PCP_QA_TEST/994
echo
echo export PATH=$PCP_INSTALL/bin:\$PATH
echo export LD_LIBRARY_PATH=$PCP_INSTALL/lib
echo export PCP_DIR=$PCP_INSTALL
echo export PYTHONPATH=$PCP_INSTALL/lib64/python$PCP_PYTHON_VER/site-packages/
echo here=$PCP_QA_TEST
echo cd \$here
echo
echo "Configuring and testing QA..."
export PATH=$PCP_INSTALL/bin:$PATH
export LD_LIBRARY_PATH=$PCP_INSTALL/lib
export PCP_DIR=$PCP_INSTALL
export PYTHONPATH=$PCP_INSTALL/lib64/python$PCP_PYTHON_VER/site-packages/
cd $PCP_QA_TEST
if [ ! -f ./localconfig ]; then
printf "y\ny\n" | ./chk.setup
fi
./check 1069
Cheers,
--
Marko Myllynen
From myllynen@redhat.com Mon Mar 7 08:15:16 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay3.corp.sgi.com [198.149.34.15])
by oss.sgi.com (Postfix) with ESMTP id BD1D87CA0
for ; Mon, 7 Mar 2016 08:15:16 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
by relay3.corp.sgi.com (Postfix) with ESMTP id 33623AC001
for ; Mon, 7 Mar 2016 06:15:12 -0800 (PST)
X-ASG-Debug-ID: 1457360108-04cbb00b70224de0001-S8gJnT
Received: from mail-wm0-f49.google.com (mail-wm0-f49.google.com [74.125.82.49]) by cuda.sgi.com with ESMTP id JG5ofIMKmS5t6bOp (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Mon, 07 Mar 2016 06:15:09 -0800 (PST)
X-Barracuda-Envelope-From: myllynen@redhat.com
X-Barracuda-Apparent-Source-IP: 74.125.82.49
Received: by mail-wm0-f49.google.com with SMTP id l68so72052002wml.1
for ; Mon, 07 Mar 2016 06:15:09 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20130820;
h=x-gm-message-state:reply-to:to:from:subject:organization:message-id
:date:user-agent:mime-version:content-transfer-encoding;
bh=D6Tm6O5bBvwNbNaVqcGSgSHP61QTUjmEc1P84ShJm2U=;
b=guB9HQfh74d/FgtRWD/Wi2oiGEwA/1U/uaLopmFrxWnUed9YUSJVJR70xHHa3GjYAK
wolTiC5oo7D6rdr06QilalI//I5fI3AD76h9MVFI1kB+lbHLXA00L4S3Cz8kUVexvdc9
/DM8RY5/Nri9KrqbkhPVcGcy5zrDjTgEIYEptJGI6/Fgi7fWElSWnaAAh8qJlb7zbXs4
W/GBgmLQ9qE8Vw9AOh1ul7tyAZD7d+XaJ6wCnE8MUmJv2napX3Dtzfsn+L0vAqXAVUTd
iH3lm86GntQLKer/bvMdJokqZ0EjB/NDVU1oUpFyRZn1T343O+wuuOl5lYAqktBRqLZ5
AHMQ==
X-Gm-Message-State: AD7BkJLMp5fJQXaqcFlRvnbAnTHuHnZWo9rzrrbr9bitWrl9GPwdUjXM1Ym8pYzU/umPBAHa
X-Received: by 10.28.226.212 with SMTP id z203mr13445896wmg.78.1457360108370;
Mon, 07 Mar 2016 06:15:08 -0800 (PST)
Received: from ?IPv6:2001:14bb:100:24e4:402:1fcc:a508:2? (dyfycl5yfyks85lkkyyby-4.rev.dnainternet.fi. [2001:14bb:100:24e4:402:1fcc:a508:2])
by smtp.gmail.com with ESMTPSA id u14sm7697916wmu.8.2016.03.07.06.15.07
for
(version=TLSv1/SSLv3 cipher=OTHER);
Mon, 07 Mar 2016 06:15:07 -0800 (PST)
Reply-To: Marko Myllynen
To: pcp developers
From: Marko Myllynen
Subject: PMDA Metrics Limit
Organization: Red Hat
X-ASG-Orig-Subj: PMDA Metrics Limit
Message-ID: <56DD8CE8.9020202@redhat.com>
Date: Mon, 7 Mar 2016 16:15:04 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
Thunderbird/38.6.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Barracuda-Connect: mail-wm0-f49.google.com[74.125.82.49]
X-Barracuda-Start-Time: 1457360109
X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256
X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
X-Barracuda-Spam-Score: 0.00
X-Barracuda-Spam-Status: No, SCORE=0.00 using per-user scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=2.7 tests=
X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.27644
Rule breakdown below
pts rule name description
---- ---------------------- --------------------------------------------------
Hi,
A while back there was a fix to remove a limit of 1024 when importing
logs:
http://oss.sgi.com/cgi-bin/gitweb.cgi?p=pcp/pcp.git;a=commit;h=90ae8bc5e69e754760a9caa3cd0a18b2de0bc503
It would seem that the limitation still exists for PMDAs, see the below
test PMDA - it works as expected if the innermost loop is removed from
generate_metrics() (676 metrics added) but if the innermost loop is in
place, then only 1024 metrics are added even though there are over 17k
metrics in the hash:
use strict;
use warnings;
use PCP::PMDA;
my $pmda;
sub test_fetch_callback {
return (PM_ERR_AGAIN, 0);
}
sub generate_metrics {
my %metrics;
for my $i ("a".."z") {
for my $j ("a".."z") {
for my $k ("a".."z") {
$metrics{$i . $j . $k} = 0;
}
}
}
$pmda->log(scalar(keys %metrics));
return %metrics;
}
$pmda = PCP::PMDA->new('test', 497);
my %data = generate_metrics() unless defined($ENV{'PCP_PERL_DOMAIN'});
my $i = 0;
foreach my $key (keys %data) {
my $name = "test" . '.' . $key;
$pmda->add_metric(pmda_pmid(0, $i++), PM_TYPE_STRING, 0,
PM_SEM_INSTANT, pmda_units(0,0,0,0,0,0),
$name, '', '');
}
$pmda->set_fetch_callback(\&test_fetch_callback);
$pmda->run();
Thanks,
--
Marko Myllynen
From fche@redhat.com Mon Mar 7 08:16:01 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=none autolearn=ham
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
by oss.sgi.com (Postfix) with ESMTP id 374767CA0
for ; Mon, 7 Mar 2016 08:16:01 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
by relay1.corp.sgi.com (Postfix) with ESMTP id 0A55B8F8033
for ; Mon, 7 Mar 2016 06:15:57 -0800 (PST)
X-ASG-Debug-ID: 1457360156-04cbb00b70224f50001-S8gJnT
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by cuda.sgi.com with ESMTP id tSmcIoZBPOeJOuR7 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 07 Mar 2016 06:15:56 -0800 (PST)
X-Barracuda-Envelope-From: fche@redhat.com
X-Barracuda-Apparent-Source-IP: 209.132.183.28
X-ASG-Whitelist: Client
Received: from int-mx11.intmail.prod.int.phx2.redhat.com (int-mx11.intmail.prod.int.phx2.redhat.com [10.5.11.24])
by mx1.redhat.com (Postfix) with ESMTPS id 4520CC0B78BA
for ; Mon, 7 Mar 2016 14:15:56 +0000 (UTC)
Received: from fche.csb (vpn-58-5.rdu2.redhat.com [10.10.58.5])
by int-mx11.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id u27EFtfQ013036;
Mon, 7 Mar 2016 09:15:56 -0500
Received: by fche.csb (Postfix, from userid 2569)
id 87EB8585D2; Mon, 7 Mar 2016 09:15:55 -0500 (EST)
To: Mark Goodwin
Cc: pcp
Subject: Re: patch/RFC - install global derived metrics dir and configs
References: <56DD0704.3000800@redhat.com>
X-ASG-Orig-Subj: Re: patch/RFC - install global derived metrics dir and configs
From: fche@redhat.com (Frank Ch. Eigler)
Date: Mon, 07 Mar 2016 09:15:55 -0500
In-Reply-To: <56DD0704.3000800@redhat.com> (Mark Goodwin's message of "Mon, 7 Mar 2016 15:43:48 +1100")
Message-ID:
User-Agent: Gnus/5.1008 (Gnus v5.10.8) Emacs/21.4 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Scanned-By: MIMEDefang 2.68 on 10.5.11.24
X-Barracuda-Connect: mx1.redhat.com[209.132.183.28]
X-Barracuda-Start-Time: 1457360156
X-Barracuda-Encrypted: ECDHE-RSA-AES256-GCM-SHA384
X-Barracuda-URL: https://192.48.176.25:443/cgi-mod/mark.cgi
X-Virus-Scanned: by bsmtpd at sgi.com
X-Barracuda-BRTS-Status: 1
mgoodwin wrote:
> RFC - ship a global derived metrics config directory in the pcp package.
> $PCP_VAR_DIR/config/derived along with a config for IOSTAT metrics. [...]
Makes perfect sense, except why $PCP_VAR_DIR as opposed to $PCP_SYSCONF_DIR ?
- FChE
From Martin@holmsgaard.com Mon Mar 7 11:45:02 2016
Return-Path:
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on oss.sgi.com
X-Spam-Level:
X-Spam-Status: No, score=0.0 required=5.0 tests=HTML_MESSAGE autolearn=ham
version=3.3.1
X-Original-To: pcp@oss.sgi.com
Delivered-To: pcp@oss.sgi.com
Received: from relay.sgi.com (relay1.corp.sgi.com [137.38.102.111])
by oss.sgi.com (Postfix) with ESMTP id 3AA417CA0
for ; Mon, 7 Mar 2016 11:45:02 -0600 (CST)
Received: from cuda.sgi.com (cuda2.sgi.com [192.48.176.25])
by relay1.corp.sgi.com (Postfix) with ESMTP id 14C5F8F8049
for ; Mon, 7 Mar 2016 09:44:58 -0800 (PST)
X-ASG-Debug-ID: 1457372690-04cbb00b71234e90001-S8gJnT
Received: from smtp-mx-server-8.servers.netregistry.net (smtp-outgoing-66.servers.netregistry.net [202.124.241.66]) by cuda.sgi.com with ESMTP id CJbfQoh0B0uWAocs (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Mon, 07 Mar 2016 09:44:51 -0800 (PST)
X-Barracuda-Envelope-From: Martin@holmsgaard.com
X-Barracuda-RBL-Trusted-Forwarder: 202.124.241.66
Received: from mail1.bemta3.messagelabs.com ([195.245.230.172])
by smtp-mx-server-8.servers.netregistry.net protocol: esmtps (Exim 4.84 #2 (Debian))
id 1aczCy-0005XC-Ln
for ; Tue, 08 Mar 2016 04:44:48 +1100
Received: from [85.158.137.68] by server-12.bemta-3.messagelabs.com id 7B/38-19343-FFDBDD65; Mon, 07 Mar 2016 17:44:31 +0000
X-Barracuda-Apparent-Source-IP: 195.245.230.172
X-Barracuda-RBL-IP: 195.245.230.172
X-Barracuda-BBL-IP: 195.245.230.172
X-Env-Sender: Martin@holmsgaard.com
X-Msg-Ref: server-13.tower-31.messagelabs.com!1457372670!26953847!1
X-Originating-IP: [130.185.138.52]
X-StarScan-Received:
X-StarScan-Version: 8.11; banners=-,-,-
X-VirusChecked: Checked
Received: (qmail 59577 invoked from network); 7 Mar 2016 17:44:31 -0000
Received: from unknown (HELO mail.holmsgaard.com) (130.185.138.52)
by server-13.tower-31.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 7 Mar 2016 17:44:31 -0000
Received: from lombard.com (85.117.96.160) by SRV02.holmsgaard.com
(192.168.4.12) with Microsoft SMTP Server id 14.3.279.2; Mon, 7 Mar 2016
18:44:27 +0100
Message-ID:
From: Iam Human
To: , ,