Re: Floating point problem

[Ken McDonell <kenj@xxxxxxxxxxxxxxxx>]

On 31/07/14 04:35, Martin Spier wrote:
> I'm not familiar with PCP's code so I might be completely wide of the
> mark here, but my understanding is that it's a precision problem. Aren't
> int64 or GMP (https://gmplib.org/) options in this case? There's also
> the JSON conversion problem, but one option in this case is sending out
> a String in the JSON response instead of a number. It's not clean and
> nice, but it might work. On the client side (JavaScript) I can convert
> that to a BigDecimal and work with it.
Hi all.

The problem is (a) the derived metric has the wrong semantics ... 
counter not delta(counter) and (b) the derived metric involves division 
... this will never produce an integer result.
I you don't use derived metrics and do all of the rate and percentage 
arithmetic on the client side (option 2. in my original mail), the 
problem goes away.
Otherwise, you need an implementation of option 1. or 3.

I've started working on 3.