pcp
[Top] [All Lists]

Re: JSON PMDA with indom cache changes

To: Nathan Scott <nathans@xxxxxxxxxx>
Subject: Re: JSON PMDA with indom cache changes
From: David Smith <dsmith@xxxxxxxxxx>
Date: Thu, 04 Jun 2015 13:30:42 -0500
Cc: pcp <pcp@xxxxxxxxxxx>
Delivered-to: pcp@xxxxxxxxxxx
In-reply-to: <2105731158.10292085.1433231032998.JavaMail.zimbra@xxxxxxxxxx>
References: <555DEF05.7030108@xxxxxxxxxx> <461406362.3814890.1432280874168.JavaMail.zimbra@xxxxxxxxxx> <55677BB7.3060805@xxxxxxxxxx> <1432466455.9615965.1433146915880.JavaMail.zimbra@xxxxxxxxxx> <2105731158.10292085.1433231032998.JavaMail.zimbra@xxxxxxxxxx>
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0
On 06/02/2015 02:43 AM, Nathan Scott wrote:
> Hi David,
> 
> ----- Original Message -----
>> [...]
>> Otherwise, looks good to me - just running a full QA run - will let you
>> know if anything unexpected happens in the new tests.
>>
> 
> Minor issues - I pushed in fixes for 201 and 1053, but still seeing some
> non-deterministic output from test 1052 (related to name hashing I guess);
> its probably safe to arrange for the dbpmda output to be sorted?  See the
> attached .bad file from the failing test.

Interesting. Here's the diff between 1052.out and 1052.out.bad:

====
diff -u qa/1052.out ~/1052.out.bad
--- qa/1052.out 2015-05-27 15:26:12.110733539 -0500
+++ /home/dsmith/1052.out.bad   2015-06-04 13:15:24.645703624 -0500
@@ -64,8 +64,8 @@
 value 2
 dbpmda> children json
 Metric: json
-non-leaf s2
 non-leaf s1
+non-leaf s2
 leaf debug
 leaf nsources
 dbpmda> children json.s1
@@ -100,15 +100,15 @@
 value 3
 dbpmda> children json
 Metric: json
-non-leaf ceph
 non-leaf s2
 non-leaf s1
+non-leaf ceph
 leaf debug
 leaf nsources
 dbpmda> children json.s1
 Metric: json.s1
-leaf string
 leaf value
+leaf string
 dbpmda> children json.s2
 Metric: json.s2
 non-leaf array_data
====

What's interesting is that I made sure that things got added in the same
order and with the same cluster numbers.j

When you say things are "non-deterministic" do you mean that sometimes
you see a pass? I certainly never saw this kind of behavior when I was
testing.

Hmm. Since we have fairly full control over commands here, we could do
something like the following (untested) patch. Basically we sort the
output of only the 'children json' dbpmda command

====
diff --git a/qa/1052 b/qa/1052
index 1d8b778..685a1fe 100755
--- a/qa/1052
+++ b/qa/1052
@@ -65,6 +65,16 @@ _filter()
     #end
 }

+_drain_output_sorted()
+{
+    quit=0
+    while [ $quit -eq 0 ]; do
+       # detect failure, but have a last round
+       read -u 4 -t 1 output || quit=1
+       echo "$output" | _filter | LC_COLLATE=POSIX sort
+    done
+}
+
 cleanup()
 {
     cd $here
@@ -217,10 +227,14 @@ open pipe $python_path $pmda_script
 getdesc on
 desc json.nsources
 fetch json.nsources
-children json
 EOF
 _drain_output

+cat >&3 <<EOF
+children json
+EOF
+_drain_output_sorted
+
 # Now, let's add a JSON data source.
 mkdir ${tmp}/s1
 cp $SRC1_METADATA ${tmp}/s1/metadata.json
====

Note there are a couple of other places that would need the same treatment.

-- 
David Smith
dsmith@xxxxxxxxxx
Red Hat
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)

<Prev in Thread] Current Thread [Next in Thread>