pcp
[Top] [All Lists]

Floating point problem

To: pcp@xxxxxxxxxxx
Subject: Floating point problem
From: Martin Spier <mspier@xxxxxxxxxxx>
Date: Fri, 25 Jul 2014 17:58:51 -0700
Cc: Coburn Watson <cwatson@xxxxxxxxxxx>, Brendan Gregg <bgregg@xxxxxxxxxxx>, Amer Ather <aather@xxxxxxxxxxx>
Delivered-to: pcp@xxxxxxxxxxx
Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netflix.com; s=google; h=mime-version:date:message-id:subject:from:to:cc:content-type; bh=H/EB8OeRXFxqqoDC7sipkMQ74qVaofp2qzfW5808WJA=; b=KjqkA8TD5htVqdrT+Wv0IUy7589PW0dcrwgNiEz3ssgTLG7FdJitjfaBazErusLbHk KDpVnE/KmwM8DCJqevhVEA/S7PP3BGTWJ3wDVHQMcPpncA39RfzJ90DC8niV/nxbL20l QJPejLjMRhcMSRyY3v0k/7VzSpblj3XU2jLmc=
Ran into a small problem with "value" floating point precision on pmwebd json responses.

Cumulative values get too big and I start having problems like this:

{
 "timestamp": {
  "s": 1406323733,
  "us": 288692
 },
 "values": [
  {
   "pmid": 2143289348,
   "name": "kernel.pct.cpu.user",
   "instances": [
    {
     "value": 7.94874E+10
    }
   ]
  }
 ]
}

Since I'm trying to calculate % usage in this case, subtracting previous value from current and normalizing by delta in time, at this point I can't get any useful information to plot due to the low precision.

Any suggestions?

-Martin
<Prev in Thread] Current Thread [Next in Thread>