none
text box data - covert to number and calculate NOT WORKING please help

    Question

  • I am trying to allow users of my application to enter numeric data in a textbox then add up to 5 amounts against 1 to verify they are the same. so I need to add convert text to numbers then subtract. the code has been working for months, no issue until someone tried entering 89.63 in tbxTicketAmount, 32.00 in tbxAmount1 and 57.63 in tbxAmount2. these amounts should equal 0 but it does not work. other decimals and/or whole numbers work fine.         

            Dim x1 As Double

            Dim amt1 As Double

            Dim amt2 As Double

            Dim amt3 As Double

            Dim amt4 As Double

            Dim amt5 As Double

            Dim bal As Double

            'set values for each result

            x1 = Val(tbxTicketAmount.Text)

            amt1 = Val(tbxAmount1.Text)

            amt2 = Val(tbxAmount2.Text)

            amt3 = Val(tbxAmount3.Text)

            amt4 = Val(tbxAmount4.Text)

            amt5 = Val(tbxAmount5.Text)

            'subtract values, must answer 0

            bal = x1 - amt1 - amt2 - amt3 - amt4 - amt5

            If bal <> 0 Then

                lblBalance.Text = "[!] Transaction does not balance"

                lblSUBMIT.Text = "[!] Transaction does not balance"

            Else


    Friday, April 21, 2017 9:12 PM

All replies

  • In a computer a single or double precision number has a certain rounding that occurs (or does not). So if you look a the value of what you think is an integer it can be 3.999999999(..) instead of 4. I suspect you dont get exactly zero you get 0.000000000000012  or something. If you view the value of bal it is not zero.

    One work around is just use a range of values for zero like this:

            Dim tolerance As Double = 0.000001
    
            If bal > -tolerance AndAlso bal < tolerance Then
    
            End If


    PS Using a decimal type may help if you just have hundreths.


    Friday, April 21, 2017 9:37 PM
  • Hi

    This sort of calculation can be tricky. When using Doubles in a calculation, results may (usually) have some trailing 'remainders' that can not easily be noticed until trying to do a comparison  - such as you are doing.

    The Double value of the subtractions you do, even though you can see the result should be zero, actually have a very small fraction remaining (due to the way a computer does these sort of calculations. So, when you try to compare to zero, there is no match. Without doing this sort of thing, your result as computed is -7.1054273576010019E-15 quite a small number, but does not equal zero.

    Here is some crude code to show one way to overcome the limitations. (there are many ways to do this).

    Option Strict On
    Option Explicit On
    Public Class Form1
        Dim tb1, tb2, tb3 As New TextBox
        Private Sub Form1_Load(sender As System.Object, e As EventArgs) Handles MyBase.Load
            tb1.Text = "89.63"
            tb2.Text = "32.00"
            tb3.Text = " 57.63"
    
            Dim result As Double = (CInt(GetDouble(tb1.Text) * 100) - CInt(GetDouble(tb3.Text) * 100) - CInt(GetDouble(tb2.Text) * 100)) / 100
        End Sub
        Function GetDouble(s As String) As Double
            Dim v As Double = 0.0
            If Double.TryParse(s, v) Then Return v
            Return 0.0
        End Function
    End Class


    Regards Les, Livingston, Scotland



    • Edited by leshay Friday, May 05, 2017 11:34 PM
    Friday, April 21, 2017 9:38 PM
  • Another way is to round out each double number to 2 places.  For example:


    bal = Math.Round(bal, 2)


    Solitaire

    Friday, April 21, 2017 11:09 PM
  • Hi iJams6,

    If you format the output, you can achieve the expected results.

     Private Sub TextBoxControl_Load(sender As Object, e As EventArgs) Handles MyBase.Load
            Dim x1 As Double
            Dim amt1 As Double
            Dim amt2 As Double
            Dim amt3 As Double
            Dim amt4 As Double
            Dim amt5 As Double
            Dim bal As Double
            TextBox1.Text = Convert.ToDouble(89.63)
            TextBox2.Text = Convert.ToDouble(32.0)
            TextBox3.Text = Convert.ToDouble(57.63)
            TextBox4.Text = Convert.ToDouble(0)
            TextBox5.Text = Convert.ToDouble(0)
            TextBox6.Text = Convert.ToDouble(0)
            'set values for each result 
            x1 = Val(TextBox1.Text)
            amt1 = Val(TextBox2.Text)
            amt2 = Val(TextBox3.Text)
            amt3 = Val(TextBox4.Text)
            amt4 = Val(TextBox5.Text)
            amt5 = Val(TextBox6.Text)
            'subtract values, must answer 0
            bal = x1 - amt1 - amt2 - amt3 - amt4 - amt5
            bal = Format(bal, 0.0)
            If bal <> 0 Then
                TextBox7.Text = "[!] Transaction does not balance"
                TextBox8.Text = "[!] Transaction does not balance"
            ElseIf bal = 0 Then
                MessageBox.Show("Value=0")
            End If
            MessageBox.Show(TextBox7.Text + TextBox8.Text)
        End Sub

    Hope it is helpful to you.

    Best regards,

    Cherry Bu


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Tuesday, April 25, 2017 3:27 AM
    Moderator