| To: | pcp@xxxxxxxxxxx |
|---|---|
| Subject: | Floating point problem |
| From: | Martin Spier <mspier@xxxxxxxxxxx> |
| Date: | Fri, 25 Jul 2014 17:58:51 -0700 |
| Cc: | Coburn Watson <cwatson@xxxxxxxxxxx>, Brendan Gregg <bgregg@xxxxxxxxxxx>, Amer Ather <aather@xxxxxxxxxxx> |
| Delivered-to: | pcp@xxxxxxxxxxx |
| Dkim-signature: | v=1; a=rsa-sha256; c=relaxed/relaxed; d=netflix.com; s=google; h=mime-version:date:message-id:subject:from:to:cc:content-type; bh=H/EB8OeRXFxqqoDC7sipkMQ74qVaofp2qzfW5808WJA=; b=KjqkA8TD5htVqdrT+Wv0IUy7589PW0dcrwgNiEz3ssgTLG7FdJitjfaBazErusLbHk KDpVnE/KmwM8DCJqevhVEA/S7PP3BGTWJ3wDVHQMcPpncA39RfzJ90DC8niV/nxbL20l QJPejLjMRhcMSRyY3v0k/7VzSpblj3XU2jLmc= |
|
Ran into a small problem with "value" floating point precision on pmwebd json responses. Cumulative values get too big and I start having problems like this: { Â "timestamp": { Â Â "s": 1406323733, Â Â "us": 288692 Â }, Â "values": [ Â Â { Â Â Â "pmid": 2143289348,
   "name": "kernel.pct.cpu.user",    "instances": [     {      "value": 7.94874E+10     }    ]
  }  ] } Since I'm trying to calculate % usage in this case, subtracting previous value from current and normalizing by delta in time, at this point I can't get any useful information to plot due to the low precision.
Any suggestions? -Martin |
| <Prev in Thread] | Current Thread | [Next in Thread> |
|---|---|---|
| ||
| Previous by Date: | RE: [pcp] Combined metrics with PCP_DERIVED_CONFIG, Ken McDonell |
|---|---|
| Next by Date: | Re: [pcp] Combined metrics with PCP_DERIVED_CONFIG, Martin Spier |
| Previous by Thread: | pcp updates: mgoodwin merge, qa, Nathan Scott |
| Next by Thread: | RE: [pcp] Floating point problem, Ken McDonell |
| Indexes: | [Date] [Thread] [Top] [All Lists] |