Hi,
when I send the numer 0.7 to both inlets of a [* ] object (yes, in correct order) the result is printed as 0.49 If I now subtract [- 0.49] the result is -2.98023e-08 While this is no major concern for me right now, I am still wondering which step in this simple calculation is triggering this. If I remember my CS lecture correctly it is the subtraction, right?
Thanks for all hints, P
Hi Peter,
On 07/01/2019 12:24, Peter P. wrote:
Hi,
when I send the numer 0.7 to both inlets of a [* ] object (yes, in correct order) the result is printed as 0.49 If I now subtract [- 0.49] the result is -2.98023e-08 While this is no major concern for me right now, I am still wondering which step in this simple calculation is triggering this. If I remember my CS lecture correctly it is the subtraction, right?
it's all the steps!
0.7 = 7/10 is not exactly representable in binary floating point, so you actually get a nearby fraction with a power of 2 as the denominator: 11744051/16777216 = 0.699999988079071044921875 0.49 = 49/100 is also not exactly representable in binary floating point, so you actually get: 2055209/4194304 = 0.4900000095367431640625 when you multiply two floating point numbers, the result is rounded to fit the available precision (for single precision float, that is 24bits): you get 16441671/33554432 = 0.4899999797344207763671875
subtracting the last two numbers (which is again rounded) reveals the behaviour of floating point that is hidden by Pd only printing 6 digits at most:
0.7*0.7-0.49 computed as float = -1/33554432 = -2.98023223876953125e-8
other environments print more digits, which shows the behaviour sooner:
ghci> (0.7 * 0.7::Float) 0.48999998
Claude
On 07.01.19 13:24, Peter P. wrote:
Hi,
when I send the numer 0.7 to both inlets of a [* ] object (yes, in correct order) the result is printed as 0.49 If I now subtract [- 0.49] the result is -2.98023e-08 While this is no major concern for me right now, I am still wondering which step in this simple calculation is triggering this. If I remember my CS lecture correctly it is the subtraction, right?
no. it's the representation of any floating point number. neither "0.7" nor "0.49" nor "x*x" (with x="0.7") can be represented *exactly* in a base-2 floating point representation. (honestly, i'm just making that up now without checking the representability of these exact numbers; however, i'm pretty sure it *is* valid though).
because both numbers are inaccurately represented, they happen to be different.
fgmasdr IOhannes
-
Claude Heiland-Allen -
IOhannes m zmoelnig -
Peter P.