none
Visual Basic doesn't calculate simple arithmetic right! RRS feed

  • Question

  • Hi, colleague,

    I use Visual Basic (Visual Studio 2017) and encountered one problem with simple arithmetic calculation. 

        Function Cal_N60(ByVal Zmi, ByVal Ni) As Double
            Dim Cri1 As Double = Calcu_Cr(Cri, Zmi)
            Cal_N60 = Ce * Cb * Cs * Cd * Cri1 * Ni
        End Function

     Ce, Cb, Cs, and Cd are all declared as public double.

    When Zmi passed as 40, Ni as 24, the Cri1 calculated is 0.9987220891752353...

    Ce=1.3, Cb=1, Cs=1.0, and Cd=1.

    VB calculate Ce * Cb * Cs * Cd * Cri1=1.3*1*1.3*1*0.9987220891752353=1.67094...

    

    The correct number should be 1.68784...

    I changed the order of variables and found VB calculated 1.3*0.9987220891752353 wrong.

    I then set Cri1=0.998722. VB calculates 1.3*0.998722=1.2983386. CORRECT!

    I then convert the called double to string, then string to value with 6 digits, it calculated wrong again.

     Anyone encountered similar problem before? What's the problem of this? Is there a way to fix it?

    Thank you very much,

    Geo

    Saturday, March 28, 2020 4:03 AM

All replies

  • Hi,
    see Double-precision floating-point formatIt is impossible to convert all decimal values (string, literal) exactly into floating point values.

    To demonstrate this, try this code:

          Dim x As Double = 8388608
          Dim y1 As Single = 0.0333
          Dim res1 As Double = x * y1
          Console.WriteLine(res1)
          '
          Dim y2 As Double = 0.0333
          Dim res2 As Double = x * y2
          Console.WriteLine(res2)


    --
    Best Regards / Viele Grüße
    Peter Fleischer (former MVP for Developer Technologies)
    Homepage, Tipps, Tricks


    Saturday, March 28, 2020 6:05 AM
  • I can't reproduce the problem.  Can you provide Calcu_Cr function?  Do you know for sure that this returns 0.998722...?

    Why haven't you typed Zmi and Ni?  You know that these will be of type 'Object', right?  It doesn't seem to be the cause of the problem, but it's bad practice to use 'Object' when a specific type is appropriate.


    Convert between VB, C#, C++, & Java (http://www.tangiblesoftwaresolutions.com)
    Instant C# - VB to C# Converter
    Instant VB - C# to VB Converter

    Saturday, March 28, 2020 1:07 PM
  • Put Option Strict On in top if your code and then correct the errors which you see. 

    Currently it is gambling code and therefore the result will as well be just a guess. 

    And try to avoid VAL it is a function we got from 'Basic' which is the same hazardous. 


    Success
    Cor


    Saturday, March 28, 2020 6:18 PM
  • Thank you, Dave,

    Here is Calcu_Cr function

       Public Function Calcu_Cr(ByVal Cri, ByVal Zij) As Double
            CR_length = (Cri + Zij) * IIf(Ob4, 0.3048, 1.0)      'CR_length in (m) 
             ' Yi (2012)
            Dim logLr As Double = Math.Log10(CR_length)
            Cr1 = -0.41 * logLr ^ 2 + 1.03 * logLr + 0.35
            If CR_length > 20 Then
                Cr1 = 1.0#
            ElseIf CR_length < 3 Then
                Cr1 = 0.75
            End If
            Calcu_Cr = Cr1 
        End Function

    Zmi=Zij=39.75, Cri=3, Ob4=true

    Ni=24


    • Edited by fea4you Sunday, March 29, 2020 2:47 AM
    Sunday, March 29, 2020 2:40 AM
  • Thank you, Cor,

    I used Val just want to try to get a 6 digits number, because when I use direct input,  

    Dim Cri1 As Double =0.988722

    the calculated value is correct.

    Fred

    Sunday, March 29, 2020 2:45 AM
  • When I use your functions and initial values, and the following test:

    Dim result As Double = Cal_N60(39.75, 24)


    I get 'result' = 30.848129182267343

    Is this not correct?

    Your functions, slightly modified to fill in the details you mentioned:

    	Public Ce As Double = 1.3
    	Public Cb As Double = 1
    	Public Cs As Double = 1.0
    	Public Cd As Double = 1
    	Function Cal_N60(ByVal Zmi As Double, ByVal Ni As Double) As Double
    		Dim Cri1 As Double = Calcu_Cr(3, Zmi)
    		Cal_N60 = Ce * Cb * Cs * Cd * Cri1 * Ni
    	End Function
    	Public Function Calcu_Cr(ByVal Cri As Double, ByVal Zij As Double) As Double
    		Dim CR_length As Double = (Cri + Zij) * IIf(True, 0.3048, 1.0)      'CR_length in (m) 
    		' Yi (2012)
    		Dim logLr As Double = Math.Log10(CR_length)
    		Dim Cr1 As Double = -0.41 * logLr ^ 2 + 1.03 * logLr + 0.35
    		If CR_length > 20 Then
    			Cr1 = 1.0#
    		ElseIf CR_length < 3 Then
    			Cr1 = 0.75
    		End If
    		Calcu_Cr = Cr1
    	End Function
    


    Convert between VB, C#, C++, & Java (http://www.tangiblesoftwaresolutions.com)
    Instant C# - VB to C# Converter
    Instant VB - C# to VB Converter

    Sunday, March 29, 2020 3:09 AM
  • HI, Dave,

    I found the problem. Cri1 should be 0.9887220891752353, not  0.9987220891752353

    Thank you again.

    Sunday, March 29, 2020 3:29 AM
  • Thank you, all.

    I found the problem. Cri1 should be 0.9887220891752353, not  0.9987220891752353

    Thank you again.

    Sunday, March 29, 2020 3:30 AM
  • Hi fea4you,

    Did you solve your problem? If your question has been answered then please click the "Mark as Answer" Link at the bottom of the correct post(s), so that it will help other members to find the solution quickly if they face a similar issue.

    Best Regards,

    Xingyu Zhao



    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Monday, March 30, 2020 2:09 AM
    Moderator