# weird math error in VB2010 • ### Question

• the following code:

x1 = 2305 / 4096

x5 = 4096

r = 2025

x1 = r / x5

results in both instances is that x1 ends up being 1, and not 0 as it should.

r, x1 and x5 are defined as int32.

solution anyone?

developer of embedded apps using VB as complement

Wednesday, April 18, 2018 9:12 PM

### All replies

• It is rounding up.  2025/4096 is greater than .5 so it becomes 1.

You will want to use Math.Floor(r/x5)

Lloyd Sheen

• Edited by Wednesday, April 18, 2018 9:49 PM
• Proposed as answer by Wednesday, April 18, 2018 11:27 PM
Wednesday, April 18, 2018 9:47 PM
• Yes, this is the rule. Rounding from a fractional or decimal number to an Integer is either up or down to the closest integer value. If the value is in middle then it's rounded up or down to the closest even integer value.

Paul ~~~~ Microsoft MVP (Visual Basic)

Wednesday, April 18, 2018 10:10 PM
• x1 = 2305/4095 makes x1 = 0.562744140625 so rounded up = 1

x1 = r / x5 makes x1 = 2025 / 4096 or 0.494384765625 which rounded down = 0.

I suppose you could use type Double or Decimal as they will contain decimal places then use Math.Floor Math.Ceiling depending on the result and how you would like to round it.

Integer data type (Visual Basic)

"Range

If you try to set a variable of an integral type to a number outside the range for that type, an error occurs. If you try to set it to a fraction, the number is rounded up or down to the nearest integer value. If the number is equally close to two integer values, the value is rounded to the nearest even integer. This behavior minimizes rounding errors that result from consistently rounding a midpoint value in a single direction. The following code shows examples of rounding."

From code "Note, Visual Basic uses banker’s rounding (toward nearest even number)"

```' The valid range of an Integer variable is -2147483648 through +2147483647.
Dim k As Integer
' The following statement causes an error because the value is too large.
k = 2147483648
' The following statement sets k to 6.
k = 5.9
' The following statement sets k to 4
k = 4.5
' The following statement sets k to 6
' Note, Visual Basic uses banker’s rounding (toward nearest even number)
k = 5.5  ```

Maybe this Function can assist you.

```Option Strict On

Public Class Form1

Dim x1 As Double = 0
Dim r As Double = 2025
Dim x5 As Double = 4096

Me.Location = New Point(CInt((Screen.PrimaryScreen.WorkingArea.Width / 2) - (Me.Width / 2)), CInt((Screen.PrimaryScreen.WorkingArea.Height / 2) - (Me.Height / 2)))
End Sub

Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
x1 = 2305 / 4096
Dim i As Integer = RoundUpOrDown(x1)
MessageBox.Show(i.ToString)
End Sub

Private Sub Button2_Click(sender As Object, e As EventArgs) Handles Button2.Click
x1 = r / x5
Dim i As Integer = RoundUpOrDown(x1)
MessageBox.Show(i.ToString)
End Sub

Private Function RoundUpOrDown(ByVal Input As Double) As Integer
Dim s As String = Input.ToString
Dim Find As Integer = 0
Dim Result As Integer = 0
If s.Contains(".") Then
Find = s.IndexOf(".") + 1
s = s.Substring(Find, 1)
MessageBox.Show(s)
Result = CInt(s)
If Result >= 5 Then
Result = CInt(Math.Ceiling(Input))
Else
Result = CInt(Math.Floor(Input))
End If
Else
Result = CInt(Input)
End If
Return Result
End Function

End Class```

La vida loca

Wednesday, April 18, 2018 11:17 PM
• the following code:

x1 = 2305 / 4096

x5 = 4096

r = 2025

x1 = r / x5

results in both instances is that x1 ends up being 1, and not 0 as it should.

r, x1 and x5 are defined as int32.

solution anyone?

developer of embedded apps using VB as complement

Hi zecharia12,

I get x1=0 according to the following code, because 2025/4096=0.49,rounding 0.

```Dim x5 As Int32 = 4096
Dim r As Int32 = 2025
Dim x1 As Int32 = r / x5
Console.WriteLine(x1)```

But x1=2305/4096=0.56, rounding 1

` Dim x1 As Int32 = 2305 / 4096`

Please check Which execution result is x1.

Best Regards,

Cherry

MSDN Community Support
Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

Thursday, April 19, 2018 7:51 AM