21 May
2021
21 May
'21
11:19 a.m.
thanks Roman for replying
i had the, apparently, wrong idea that it were the big numbers giving problems when using floating point arithmetic.
now, after reading up about the 'perils of floating point' i have to rethink my method of calculation.
rolf
Roman Haefeli wrote:
The intricacies of 32bit floating point numbers have been discussed up
and down on this list.
tldr; many decimal numbers do not have an exact floating point representation. Incrementing with such numbers increases the error on each iteration
Suggestion: Count with integer numbers and scale afterwards.
Roman