locked
Double calculation problem in C# RRS feed

  • Question

  • I have calculation statement in C# as below statement:

    (-0.0116 * (5.08 - 0.52 * Math.Pow(5.08, 1.3)) * Math.Pow(0.5, 1.1) * (-0.14 * Math.Pow((19000 * 0.5), 0.8)))

    When I debug and add this statement to watch, it was shows as below screenshot:

    but when assign to double variable, the result was changed:



    How to get 0.89713135944117184 without change any implementation

    • Edited by loonloon Friday, January 16, 2015 10:16 AM
    Friday, January 16, 2015 10:15 AM

Answers

  • You're using doubles which have precision of 15 to 16 digits. You have clearly exceeded that precision in your answer so the last couple of digits are potentially wrong because of rounding or truncation. If you need more than 15 digits of precision then you need to switch to decimal.

    Michael Taylor
    http://blogs.msmvps.com/p3net

    • Proposed as answer by Blackwood Friday, January 16, 2015 7:18 PM
    • Marked as answer by Caillen Sunday, January 25, 2015 9:07 AM
    Friday, January 16, 2015 6:46 PM

All replies

  • I get 0.89713135944117184 if I just copy your line 

    (-0.0116 * (5.08 - 0.52 * Math.Pow(5.08, 1.3)) * Math.Pow(0.5, 1.1) * (-0.14 * Math.Pow((19000 * 0.5), 0.8)))

    You may want to check some of your intermediate variables; check you're not accidentally using a float rather than a double somewhere.

    Friday, January 16, 2015 10:40 AM
  • hi,

    I have copied the same code like below.

     double temp = (-0.0116*(5.08 - 0.52*Math.Pow(5.08, 1.3))*Math.Pow(0.5, 1.1)*
                              (-0.14*Math.Pow((19000*0.5), 0.8)));

    out put I get is

    0.89713135944117184

    Friday, January 16, 2015 10:55 AM
  • I am getting 0.89713135944117184 in C++ version your code with VS 2013.

    Try add suffix of float/double whatever after the value 19000 like 19000.0f or just 19000.0 (double) to see if there is difference.

    Not sure if this thread is related to the issue you mentioned:

    Precision Problem with visual studio 2010


    Thanks & Regards

    Friday, January 16, 2015 1:38 PM
  • You're using doubles which have precision of 15 to 16 digits. You have clearly exceeded that precision in your answer so the last couple of digits are potentially wrong because of rounding or truncation. If you need more than 15 digits of precision then you need to switch to decimal.

    Michael Taylor
    http://blogs.msmvps.com/p3net

    • Proposed as answer by Blackwood Friday, January 16, 2015 7:18 PM
    • Marked as answer by Caillen Sunday, January 25, 2015 9:07 AM
    Friday, January 16, 2015 6:46 PM