On 09/08/2014 11:27 PM, Frank Ch. Eigler wrote:
Hi -
On Mon, Sep 08, 2014 at 11:06:25PM +1000, Mark Goodwin wrote:
..
[...] The following one character patch in libpcp promotes it to
double precision, [...]
- dbl += (double)val->tv_usec / 1000000.0;
+ dbl += (double)val->tv_usec / 1000000.0L;
Please note that in the L suffix means "long double".
Unsuffixed already means "double".
yes quite correct, as per http://en.wikipedia.org/wiki/Long_double
After a little bit more investigation, it seems that even double
precision arithmetic is still incurring loss of precision for
intermediate results in some (but not all) timeval to double
conversions using __pmtimevalToReal(), depending on the operands.
Suffixing L as in the patch, promotes the intermediate results to
long double arithmetic (96bits on i386 and 128 bits on x86_64), which
seems to provide enough precision to match the hand calculated expected
values, and the QA test now passes - and even more importantly pmval
now reports the correct values without the yucky rounding errors.
So I'll push the patch if there are no objections
thanks for your insight Frank :)
|