Re: Floating point problem

["Frank Ch. Eigler" <fche@xxxxxxxxxx>]

Hi -

> I'm not familiar with PCP's code so I might be completely wide of
> the mark here, but my understanding is that it's a precision
> problem. Aren't int64 or GMP (https://gmplib.org/) options in this
> case?

Doing derived-metric calculations in int64 form sounds possible, but
effort would have to be expended to define casting and rules, and
given pure C libpcp, some unpleasant code duplication would be needed.


> There's also the JSON conversion problem, but one option in this
> case is sending out a String in the JSON response instead of a
> number. [...]

pmwebd was using the equivalent of printf("%g"), which suffers from a
too-low default precision.  The fche/pmwebd branch switches to a
type-appropriate number of digits of precision, so this part should be
better.  But even with that, one will have only 52 bits bits of
precision/mantissa, fewer than with a 64-bit integer.

- FChE