# Visual Basic doesn't calculate simple arithmetic right! • ### Question

• Hi, colleague,

I use Visual Basic (Visual Studio 2017) and encountered one problem with simple arithmetic calculation.

```    Function Cal_N60(ByVal Zmi, ByVal Ni) As Double
Dim Cri1 As Double = Calcu_Cr(Cri, Zmi)
Cal_N60 = Ce * Cb * Cs * Cd * Cri1 * Ni
End Function```

Ce, Cb, Cs, and Cd are all declared as public double.

When Zmi passed as 40, Ni as 24, the Cri1 calculated is 0.9987220891752353...

Ce=1.3, Cb=1, Cs=1.0, and Cd=1.

VB calculate Ce * Cb * Cs * Cd * Cri1=1.3*1*1.3*1*0.9987220891752353=1.67094...

﻿ The correct number should be 1.68784...

I changed the order of variables and found VB calculated 1.3*0.9987220891752353 wrong. I then set Cri1=0.998722. VB calculates 1.3*0.998722=1.2983386. CORRECT! I then convert the called double to string, then string to value with 6 digits, it calculated wrong again. Anyone encountered similar problem before? What's the problem of this? Is there a way to fix it?

Thank you very much,

Geo

Saturday, March 28, 2020 4:03 AM

### All replies

• Hi,
see Double-precision floating-point formatIt is impossible to convert all decimal values (string, literal) exactly into floating point values.

To demonstrate this, try this code:

```      Dim x As Double = 8388608
Dim y1 As Single = 0.0333
Dim res1 As Double = x * y1
Console.WriteLine(res1)
'
Dim y2 As Double = 0.0333
Dim res2 As Double = x * y2
Console.WriteLine(res2)```

--
Best Regards / Viele Grüße
Peter Fleischer (former MVP for Developer Technologies)
Homepage, Tipps, Tricks

Saturday, March 28, 2020 6:05 AM
• I can't reproduce the problem.  Can you provide Calcu_Cr function?  Do you know for sure that this returns 0.998722...?

Why haven't you typed Zmi and Ni?  You know that these will be of type 'Object', right?  It doesn't seem to be the cause of the problem, but it's bad practice to use 'Object' when a specific type is appropriate.

Convert between VB, C#, C++, & Java (http://www.tangiblesoftwaresolutions.com)
Instant C# - VB to C# Converter
Instant VB - C# to VB Converter

Saturday, March 28, 2020 1:07 PM
• Put Option Strict On in top if your code and then correct the errors which you see.

Currently it is gambling code and therefore the result will as well be just a guess.

And try to avoid VAL it is a function we got from 'Basic' which is the same hazardous.

Success
Cor

Saturday, March 28, 2020 6:18 PM
• Thank you, Dave,

Here is Calcu_Cr function

```   Public Function Calcu_Cr(ByVal Cri, ByVal Zij) As Double
CR_length = (Cri + Zij) * IIf(Ob4, 0.3048, 1.0)      'CR_length in (m)
' Yi (2012)
Dim logLr As Double = Math.Log10(CR_length)
Cr1 = -0.41 * logLr ^ 2 + 1.03 * logLr + 0.35
If CR_length > 20 Then
Cr1 = 1.0#
ElseIf CR_length < 3 Then
Cr1 = 0.75
End If
Calcu_Cr = Cr1
End Function```

Zmi=Zij=39.75, Cri=3, Ob4=true

Ni=24

• Edited by Sunday, March 29, 2020 2:47 AM
Sunday, March 29, 2020 2:40 AM
• Thank you, Cor,

I used Val just want to try to get a 6 digits number, because when I use direct input,

Dim Cri1 As Double =0.988722

the calculated value is correct.

Fred

Sunday, March 29, 2020 2:45 AM
• When I use your functions and initial values, and the following test:

`Dim result As Double = Cal_N60(39.75, 24)`

I get 'result' = 30.848129182267343

Is this not correct?

Your functions, slightly modified to fill in the details you mentioned:

```	Public Ce As Double = 1.3
Public Cb As Double = 1
Public Cs As Double = 1.0
Public Cd As Double = 1
Function Cal_N60(ByVal Zmi As Double, ByVal Ni As Double) As Double
Dim Cri1 As Double = Calcu_Cr(3, Zmi)
Cal_N60 = Ce * Cb * Cs * Cd * Cri1 * Ni
End Function
Public Function Calcu_Cr(ByVal Cri As Double, ByVal Zij As Double) As Double
Dim CR_length As Double = (Cri + Zij) * IIf(True, 0.3048, 1.0)      'CR_length in (m)
' Yi (2012)
Dim logLr As Double = Math.Log10(CR_length)
Dim Cr1 As Double = -0.41 * logLr ^ 2 + 1.03 * logLr + 0.35
If CR_length > 20 Then
Cr1 = 1.0#
ElseIf CR_length < 3 Then
Cr1 = 0.75
End If
Calcu_Cr = Cr1
End Function
```

Convert between VB, C#, C++, & Java (http://www.tangiblesoftwaresolutions.com)
Instant C# - VB to C# Converter
Instant VB - C# to VB Converter

Sunday, March 29, 2020 3:09 AM
• HI, Dave,

I found the problem. Cri1 should be 0.9887220891752353, not  0.9987220891752353

Thank you again.

Sunday, March 29, 2020 3:29 AM
• Thank you, all.

I found the problem. Cri1 should be 0.9887220891752353, not  0.9987220891752353

Thank you again.

Sunday, March 29, 2020 3:30 AM
• Hi fea4you,

Did you solve your problem? If your question has been answered then please click the "Mark as Answer" Link at the bottom of the correct post(s), so that it will help other members to find the solution quickly if they face a similar issue.

Best Regards,

Xingyu Zhao

MSDN Community Support
Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

Monday, March 30, 2020 2:09 AM