none
What is wrong with my math? RRS feed

  • Question

  •  [TestMethod]
     public void Test()
     {
          var b = .1;
          var a = .2;
           bool c = ((a + b) == 0.3);
           Assert.IsTrue(c, "No math");
     }

    What am I doing wrong?

    Thursday, May 31, 2018 7:53 PM

Answers

All replies

  • A common issue with floating-point arithmetic, that is not as precise as you might think. 'What Every Computer Scientist Should Know About Floating-Point Arithmetic" is a good read if you're looking for explanations.

    wizend

    • Marked as answer by Arne Garvan Thursday, May 31, 2018 11:18 PM
    Thursday, May 31, 2018 8:10 PM
  • Greetings Arne.

    In case you don't want to go through the whole article in Wizend's link, here's the Readers' Digest version.

    It's impossible for a computer to store floating point numbers (numbers with a decimal part, or to put it another way, not integers) to infinite accuracy. This means that when you perform mathematical operations on floating point numbers, you often get teensy-weensy errors, even when it seems obvious to you what the result should be.

    So in your example above, (a+b) is probably 0.29999999999 or 3.00000001 or something like that. This means that when comparing floats, you need to use a tolerance, rather than test that they are exactly equal.

     public void Test()
     {
          var b = .1;
          var a = .2;
          bool c = NearEqual((a + b), 0.3, 0.001);
          Assert.IsTrue(c, "No math");
     }
    
    bool NearEqual(double x, double y, float tolerance)
    {
       return ((x > y - tolerance) && (x < y + tolerance));
    }

    Thursday, May 31, 2018 11:24 PM
  • Drat. Wasn't quick enough.
    Thursday, May 31, 2018 11:25 PM
  • I want to pipe in too, because this is an interesting topic.

    We see 0.1 and 0.2 as terminating decimals, but that's only because we are trained to use base 10.  In base 2, the only "terminating decimals" are the powers of 2.  So, 0.5 and 0.75 can be represented exactly in binary, but representing 1/10 in base 2 is like representing 1/3 in base 10.  With a fixed number of digits, you cannot represent it exactly.  Your computation gets a different rounding error than the compiler's constant for 3/10.


    Tim Roberts, Driver MVP Providenza & Boekelheide, Inc.

    Friday, June 1, 2018 12:46 AM