thanks Roman for replying
i had the, apparently, wrong idea that it were the big numbers giving problems when using floating point arithmetic.
now, after reading up about the 'perils of floating point' i have to rethink my method of calculation.
rolf
Roman Haefeli wrote:
The intricacies of 32bit floating point numbers have been discussed up
and down on this list.
tldr; many decimal numbers do not have an exact floating point representation. Incrementing with such numbers increases the error on each iteration
Suggestion: Count with integer numbers and scale afterwards.
Roman
On Fri, 2021-05-21 at 11:19 +0200, rolfm@dds.nl wrote:
i had the, apparently, wrong idea that it were the big numbers giving problems when using floating point arithmetic.
No, your idea is not wrong. Large numbers are a concern, too. Only integer numbers up to 16777216 are properly represented by 32-bit floating point numbers. Above that, a counter incrementing by 1 is stuck.
Roman
-
rolfm＠dds.nl -
Roman Haefeli