Hi Joe
Pd uses 32 bit floating point number format which has (as any other number format) some limitations. Not every number can be exactly expressed by this format. '7.1' seems to be such a number.
In case you work with only one digit after the decimal point, I suggest to multiply your number inputs by 10 so that they are all integers. Then perform your calculations in this scale. In the end you can downscale your results again with [/ 10]. The reason is that all integers up to 2^23 (or was it 2^24?) can be represented _exactly_ by 32 bit float.
BTW: This topic has extensively been discussed several times on this list. You may find more information in the Pd-list archives.
Roman
On Thu, 2011-12-08 at 00:13 -0800, Joe Newlin wrote:
Can someone explain what's going on in the attached example? I'm getting 7.1 minus 7 equals .099. I need to split two-digit integers, so this result is giving me problems.
Thanks,
JN
-- www.joenewlin.net www.twitter.com/joe_newlin
Pd-list@iem.at mailing list UNSUBSCRIBE and account-management -> http://lists.puredata.info/listinfo/pd-list