On 07.01.19 13:24, Peter P. wrote:
Hi,
when I send the numer 0.7 to both inlets of a [* ] object (yes, in correct order) the result is printed as 0.49 If I now subtract [- 0.49] the result is -2.98023e-08 While this is no major concern for me right now, I am still wondering which step in this simple calculation is triggering this. If I remember my CS lecture correctly it is the subtraction, right?
no. it's the representation of any floating point number. neither "0.7" nor "0.49" nor "x*x" (with x="0.7") can be represented *exactly* in a base-2 floating point representation. (honestly, i'm just making that up now without checking the representability of these exact numbers; however, i'm pretty sure it *is* valid though).
because both numbers are inaccurately represented, they happen to be different.
fgmasdr IOhannes