why when i sub 0.28 from 029 i get wrong answer ??
-
Monday, April 30, 2012 4:34 PM
MessageBox.Show((0.28-0.29).ToString());
i get (-0.00999999999999995) rather than -0.01
All Replies
-
Monday, April 30, 2012 4:38 PM
http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems
Yay computer science! This is a problem with doubles and floats always.
If you want to only show the accuracy to a certain point, you can try using the string formatter (Probably ToString("0.00"))
If you rather need to tell that the difference is -0.01, you should probably make a function that checks whether or not the difference between two numbers is relatively small. This is pretty hard to do well, though:http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm
Another way is to store the value as an integer, just interpret it as "number of 1/100" (its like saying something is 100 pennies rather that $1.00). You'll never get more accurate that 1/100, and your max value will only be ~+/-2^24 but you'll always have an exact value which can be used for comparison.
You can also use the decimal type instead of doubles:
http://msdn.microsoft.com/en-us/library/system.decimal%28v=vs.80%29.aspx
Which is still possible to have the same type of error, the instance is just a lot smaller. However, it is a 16 byte data type, so it takes up a lot more memory and is a lot slower to use.
- Edited by Idea Hat Monday, April 30, 2012 4:51 PM
- Marked As Answer by Bob ShenMicrosoft Contingent Staff, Moderator Wednesday, May 09, 2012 2:40 AM

