locked
Arithmetic operation result in an overflow. RRS feed

  • Question

  • This error is occuring for me I have a image compression/decompression script, I'm converting to VB.
    When compressing image.
    Arithmetic operatioin result in an overflow.

        If val > BeginValue Then
                    direction__1 = Direction.Up
                    ConstValue = CShort(val - BeginValue)
                ElseIf val < BeginValue Then
                    direction__1 = Direction.Down
                    ConstValue = CShort(val - BeginValue)  <--Arithmetic operation
                ElseIf val = BeginValue Then
                    direction__1 = Direction.Straight
                    ConstValue = 0
                End If
    
                pixel.NextPixel()
                count += 1
            Else
                Return
            End If


    Original C# Code

     if (val > BeginValue)
                    {
                        direction = Direction.Up;
                        ConstValue = (short)(val - BeginValue);
                    }
                    else if (val < BeginValue)
                    {
                        direction = Direction.Down;
                        ConstValue = (short)(val - BeginValue);
                    }
                    else if (val == BeginValue)
                    {
                        direction = Direction.Straight;
                        ConstValue = 0;
                    }
    
                    pixel.NextPixel();
                    count++;
                }
                else
                {
                    return;
                }



    • Edited by AndyjDel Sunday, February 3, 2013 2:16 PM
    Sunday, February 3, 2013 12:58 PM

Answers

  • You should debug the calculation as it produces value that is outside of value expected by CShort. CShort works differently than C# casting as in C# you inform the compiler that you want to do the conversion although you might lose some information.

    If you run following codes you can see the difference.

    ' VB conversion (exceptional case)
    Module Module1
    
        Sub Main()
            Dim value As Int32 = 500000
            Dim beginValue As Int32 = 100000
            Dim result As Int32 = value - beginValue
            Dim s As Int16 = CShort(result)
            Console.WriteLine(s)
        End Sub
    
    End Module
    
    /// C# cast
    static void Main(string[] args)
    {
        int val = 500000;
        int beginValue = 100000;
        int result = val - beginValue;
        short s = (short)result;
    
        Console.WriteLine(s);
    }



    • Edited by MasaSam Sunday, February 3, 2013 4:12 PM
    • Marked as answer by Youen Zen Friday, March 1, 2013 7:50 AM
    Sunday, February 3, 2013 1:30 PM
  • Hi,

    Which version of VB or VB.Net are you using?

    I guess the error may be in the line with CShort

    or are you getting the error in another line?

    This error occurs when a value from a calculation or
    operation produces a value too big for the variable size.


    Byte can hold values 0 to 255 or (2 ^ 8) - 1

    Short or Int16 values -32768 to 32767
     the higher number = (( 2 ^ 16 ) /2) - 1

    Integer or Int32 values -2147483648 to 2147483647
    the higher number = (( 2 ^ 32 ) /2) - 1

    Long or Int64 values -18446744073709551616 to 18446744073709551615
    the higher number = (( 2 ^ 64 ) /2) - 1

    I hope this helps.  :-)


    Regards,

    profile for John Anthony Oliver at Stack Overflow, Q&A for professional and enthusiast programmers

    Click this link to see the NEW way of how to insert a picture into a forum post.

    Installing VB6 on Windows 7

    App Hub for Windows Phone & XBOX 360 developers.

    • Marked as answer by Youen Zen Friday, March 1, 2013 7:51 AM
    Sunday, February 3, 2013 1:37 PM
  • To test, I took your code and removed anything that I thought had no effect on your problem, took a form and added a button. Here is the code:

    Public Class Form1
        Dim constValue As Integer
    
        Private Sub test(val As Integer, beginValue As Integer)
            Try
                If val > beginValue Then
                    constValue = CShort(val - beginValue)
                ElseIf val < beginValue Then
                    constValue = CShort(val - beginValue)  '<--Arithmetic operation
                ElseIf val = beginValue Then
                    constValue = 0
                End If
            Catch ex As Exception
                MessageBox.Show("An error occurred" & vbNewLine & ex.Message & vbNewLine & "Debug Info: " & _
                                "Val = " & val.ToString & vbNewLine & "beginValue = " & beginValue.ToString & vbNewLine & _
                                "Val-beginValue = " & (val - beginValue).ToString & vbNewLine & _
                                "Short Maximum = " & Short.MaxValue.ToString & " Short Minimum = " & Short.MinValue.ToString)
            End Try
        End Sub
        Private Sub Button1_Click(sender As System.Object, e As System.EventArgs) Handles Button1.Click
            test(30, 200000)
        End Sub
    End Class
    

    Clicking the button gives this:

    The IDE will also show you the values of your variables when the error is displayed if you just hover  the mouse.

    • Marked as answer by Youen Zen Friday, March 1, 2013 7:50 AM
    Sunday, February 3, 2013 3:11 PM

All replies

  • You should debug the calculation as it produces value that is outside of value expected by CShort. CShort works differently than C# casting as in C# you inform the compiler that you want to do the conversion although you might lose some information.

    If you run following codes you can see the difference.

    ' VB conversion (exceptional case)
    Module Module1
    
        Sub Main()
            Dim value As Int32 = 500000
            Dim beginValue As Int32 = 100000
            Dim result As Int32 = value - beginValue
            Dim s As Int16 = CShort(result)
            Console.WriteLine(s)
        End Sub
    
    End Module
    
    /// C# cast
    static void Main(string[] args)
    {
        int val = 500000;
        int beginValue = 100000;
        int result = val - beginValue;
        short s = (short)result;
    
        Console.WriteLine(s);
    }



    • Edited by MasaSam Sunday, February 3, 2013 4:12 PM
    • Marked as answer by Youen Zen Friday, March 1, 2013 7:50 AM
    Sunday, February 3, 2013 1:30 PM
  • Hi,

    Which version of VB or VB.Net are you using?

    I guess the error may be in the line with CShort

    or are you getting the error in another line?

    This error occurs when a value from a calculation or
    operation produces a value too big for the variable size.


    Byte can hold values 0 to 255 or (2 ^ 8) - 1

    Short or Int16 values -32768 to 32767
     the higher number = (( 2 ^ 16 ) /2) - 1

    Integer or Int32 values -2147483648 to 2147483647
    the higher number = (( 2 ^ 32 ) /2) - 1

    Long or Int64 values -18446744073709551616 to 18446744073709551615
    the higher number = (( 2 ^ 64 ) /2) - 1

    I hope this helps.  :-)


    Regards,

    profile for John Anthony Oliver at Stack Overflow, Q&A for professional and enthusiast programmers

    Click this link to see the NEW way of how to insert a picture into a forum post.

    Installing VB6 on Windows 7

    App Hub for Windows Phone & XBOX 360 developers.

    • Marked as answer by Youen Zen Friday, March 1, 2013 7:51 AM
    Sunday, February 3, 2013 1:37 PM
  • I'm using VB .NET 2010.

    Oliver as in OP.
    VB after conversion

    ConstValue = CShort(val - BeginValue)  <--Arithmetic operation error

    C# before VB conversion

    ConstValue = (short)(val - BeginValue);

    Sam the link you provided is not a available.
    What would be equivalent to C# casting short in VB, Short in VB doesn't produce a expression.




    • Edited by AndyjDel Sunday, February 3, 2013 2:15 PM
    Sunday, February 3, 2013 2:11 PM
  • Oliver can you show me how this code will look? Do I need use a 32 bit integer Long? if so can you show me how to write this expression? I've CLng as well same outcome
    • Edited by AndyjDel Sunday, February 3, 2013 2:59 PM
    Sunday, February 3, 2013 2:44 PM
  • To test, I took your code and removed anything that I thought had no effect on your problem, took a form and added a button. Here is the code:

    Public Class Form1
        Dim constValue As Integer
    
        Private Sub test(val As Integer, beginValue As Integer)
            Try
                If val > beginValue Then
                    constValue = CShort(val - beginValue)
                ElseIf val < beginValue Then
                    constValue = CShort(val - beginValue)  '<--Arithmetic operation
                ElseIf val = beginValue Then
                    constValue = 0
                End If
            Catch ex As Exception
                MessageBox.Show("An error occurred" & vbNewLine & ex.Message & vbNewLine & "Debug Info: " & _
                                "Val = " & val.ToString & vbNewLine & "beginValue = " & beginValue.ToString & vbNewLine & _
                                "Val-beginValue = " & (val - beginValue).ToString & vbNewLine & _
                                "Short Maximum = " & Short.MaxValue.ToString & " Short Minimum = " & Short.MinValue.ToString)
            End Try
        End Sub
        Private Sub Button1_Click(sender As System.Object, e As System.EventArgs) Handles Button1.Click
            test(30, 200000)
        End Sub
    End Class
    

    Clicking the button gives this:

    The IDE will also show you the values of your variables when the error is displayed if you just hover  the mouse.

    • Marked as answer by Youen Zen Friday, March 1, 2013 7:50 AM
    Sunday, February 3, 2013 3:11 PM
  • To test, I took your code and removed anything that I thought had no effect on your problem, took a form and added a button. Here is the code:

    Public Class Form1
        Dim constValue As Integer
    
        Private Sub test(val As Integer, beginValue As Integer)
            Try
                If val > beginValue Then
                    constValue = CShort(val - beginValue)
                ElseIf val < beginValue Then
                    constValue = CShort(val - beginValue)  '<--Arithmetic operation
                ElseIf val = beginValue Then
                    constValue = 0
                End If
            Catch ex As Exception
                MessageBox.Show("An error occurred" & vbNewLine & ex.Message & vbNewLine & "Debug Info: " & _
                                "Val = " & val.ToString & vbNewLine & "beginValue = " & beginValue.ToString & vbNewLine & _
                                "Val-beginValue = " & (val - beginValue).ToString & vbNewLine & _
                                "Short Maximum = " & Short.MaxValue.ToString & " Short Minimum = " & Short.MinValue.ToString)
            End Try
        End Sub
        Private Sub Button1_Click(sender As System.Object, e As System.EventArgs) Handles Button1.Click
            test(30, 200000)
        End Sub
    End Class

    Clicking the button gives this:

    The IDE will also show you the values of your variables when the error is displayed if you just hover  the mouse.

    What needs to be done to fix this? I'm not familar with Data types.
    Sunday, February 3, 2013 3:14 PM
  • Note:

    If val > BeginValue then constvalue = val-beginvalue

    If val < BeginValue then constvalue = val-beginvalue

    If val = BeginValue then constvalue = 0 which is the same as val-beginvalue

    so for all values of val and beginvalue, the result is always val-beginval, it might make your code more readable later to move that out of the conditionals block.

    Sunday, February 3, 2013 3:17 PM
  • another alternative is this, at the very beginning of the Sub:

            If Short.TryParse((val - beginValue), constValue) = False Then
                MessageBox.Show("The expression 'val - beginvalue' cannot be converted to a Short" & vbNewLine & _
                                "the result was " & (val - beginValue).ToString & vbNewLine & _
                                "Short Maximum = " & Short.MaxValue.ToString & " Short Minimum = " & Short.MinValue.ToString)
                Exit Sub ' or whatever you need to do in this case
            End If
    

    Sunday, February 3, 2013 3:31 PM
  • If the output is exceeding  the limit CShort -32768 to 32767. I tried CLng and I still get the same error.

     ConstValue = CLng(val - BeginValue)

    Here's the whole code block.

    Imports System.Drawing.Imaging
    
    Public Class ReCreator
    
        Private Enum TargetChannel
            A
            R
            G
            B
        End Enum
        Private Enum Direction
            Down
            Up
            Straight
        End Enum
    
    
        Public Sub New()
        End Sub
    
        Public Function Compress(ByVal bitmap As Bitmap) As Byte()
            If bitmap.Width = 1 AndAlso bitmap.Height = 1 Then
                Throw New Exception("Image is too small to compress")
            End If
    
            Dim payload As New PayloadWriter()
            payload.WriteUShort(CUShort(bitmap.Width))
            payload.WriteUShort(CUShort(bitmap.Height))
            payload.WriteInteger(CInt(bitmap.PixelFormat))
    
            Dim data As BitmapData = bitmap.LockBits(New Rectangle(0, 0, bitmap.Width, bitmap.Height), System.Drawing.Imaging.ImageLockMode.[ReadOnly], bitmap.PixelFormat)
            Dim ptr As IntPtr = data.Scan0
            'Dim pixel As New Pixel(bitmap.PixelFormat, CType(ptr, PointerToByte))
            Dim pixel As New Pixel(bitmap.PixelFormat, New PointerToByte(data.Scan0))
            Dim target As TargetChannel = If(pixel.isARGB, TargetChannel.A, TargetChannel.R)
            Dim Loops As Integer = bitmap.Width * bitmap.Height
            'just for not re-calculating everytime in the loop
            Dim Targets As Integer = If(pixel.isARGB, 4, 3)
    
            For j As Integer = 0 To Targets - 1
                Dim BitsPw As New PayloadWriter()
                Dim ChannelPw As New PayloadWriter()
                pixel.point = New PointerToByte(data.Scan0)
                'reset pointer
                Dim bit As New Bit()
                Dim BitIndex As Integer = 0
    
                For i As Integer = 0 To Loops - 1
                    Dim count As Byte = 1
                    Dim BeginValue As Byte = 0
                    Dim ConstValue As Short = 0
                    FindPattern(i, Loops, pixel, target, ConstValue, count, _
                     BeginValue)
    
                    If ConstValue < 0 Then
                        ConstValue = CShort(-ConstValue)
                        bit.SetIndex(System.Math.Max(System.Threading.Interlocked.Increment(BitIndex), BitIndex - 1), False)
                    ElseIf ConstValue > 0 Then
                        bit.SetIndex(System.Math.Max(System.Threading.Interlocked.Increment(BitIndex), BitIndex - 1), True)
                    End If
    
                    ChannelPw.WriteByte(CByte(ConstValue))
                    ChannelPw.WriteByte(count)
                    ChannelPw.WriteByte(BeginValue)
    
                    If BitIndex = 8 Then
                        BitsPw.WriteByte(bit.ToByte())
                        bit = New Bit()
                        BitIndex = 0
                    End If
                Next
    
                If BitIndex > 0 Then
                    BitsPw.WriteByte(bit.ToByte())
                End If
    
                payload.WriteInteger(CInt(BitsPw.Length))
                payload.WriteBytes(BitsPw.ToByteArray())
                payload.WriteInteger(CInt(ChannelPw.Length))
                payload.WriteBytes(ChannelPw.ToByteArray())
                target += 1
            Next
            pixel = Nothing
            bitmap.UnlockBits(data)
            Return payload.ToByteArray()
        End Function
    
        Private Sub FindPattern(ByRef curLoops As Integer, ByVal MaxLoops As Integer, ByRef pixel As Pixel, ByVal target As TargetChannel, ByRef ConstValue As Short, ByRef count As Byte, _
         ByRef BeginValue As Byte)
            Dim direction__1 As Direction = Direction.Straight
            Dim val As Byte = 0
    
            Select Case target
                Case TargetChannel.A
                    BeginValue = pixel.A
                    Exit Select
                Case TargetChannel.R
                    BeginValue = pixel.R
                    Exit Select
                Case TargetChannel.G
                    BeginValue = pixel.G
                    Exit Select
                Case TargetChannel.B
                    BeginValue = pixel.B
                    Exit Select
            End Select
    
            ConstValue = 0
            curLoops += 1
    
            If curLoops < MaxLoops Then
                pixel.NextPixel()
    
                Select Case target
                    Case TargetChannel.A
                        val = pixel.A
                        Exit Select
                    Case TargetChannel.R
                        val = pixel.R
                        Exit Select
                    Case TargetChannel.G
                        val = pixel.G
                        Exit Select
                    Case TargetChannel.B
                        val = pixel.B
                        Exit Select
                End Select
    
                If val > BeginValue Then
                    direction__1 = Direction.Up
                    ConstValue = CShort(val - BeginValue)
                ElseIf val < BeginValue Then
                    direction__1 = Direction.Down
                    ConstValue = CShort(val - BeginValue)
                ElseIf val = BeginValue Then
                    direction__1 = Direction.Straight
                    ConstValue = 0
                End If
    
                pixel.NextPixel()
                count += 1
            Else
                Return
            End If
    
            Dim LastValue As Short = val
            While curLoops < MaxLoops
                If count = 255 Then
                    Return
                End If
    
                LastValue += ConstValue
                Select Case target
                    Case TargetChannel.A
                        val = pixel.A
                        Exit Select
                    Case TargetChannel.R
                        val = pixel.R
                        Exit Select
                    Case TargetChannel.G
                        val = pixel.G
                        Exit Select
                    Case TargetChannel.B
                        val = pixel.B
                        Exit Select
                End Select
    
                'check if value is incremental, decremental, or just same value
                Select Case direction__1
                    Case Direction.Up, Direction.Down
                        If True Then
                            If val <> CByte(LastValue) Then
                                Return
                            End If
                            Exit Select
                        End If
                    Case Direction.Straight
                        If True Then
                            If val <> CByte(LastValue) Then
                                Return
                            End If
                            Exit Select
                        End If
                End Select
                count += 1
                pixel.NextPixel()
                curLoops += 1
            End While
        End Sub
    
        Public Function Decompress(ByVal data As Byte()) As Bitmap
            Dim payload As New PayloadReader(data)
            Dim width As Integer = payload.ReadUShort()
            Dim height As Integer = payload.ReadUShort()
            Dim format As PixelFormat = CType(payload.ReadInteger(), PixelFormat)
            Dim bitmap As New Bitmap(width, height, format)
            Dim bmpData As BitmapData = bitmap.LockBits(New Rectangle(0, 0, bitmap.Width, bitmap.Height), System.Drawing.Imaging.ImageLockMode.[WriteOnly], bitmap.PixelFormat)
            'Dim pixel As New Pixel(bitmap.PixelFormat, CType(bmpData.Scan0.ToPointer(), PointerToByte))
    
            Dim pixel As New Pixel(bitmap.PixelFormat, New PointerToByte(bmpData.Scan0))
            Dim target As TargetChannel = If(pixel.isARGB, TargetChannel.A, TargetChannel.R)
            Dim Targets As Integer = If(pixel.isARGB, 4, 3)
    
            For j As Integer = 0 To Targets - 1
                Dim BitsPr As New PayloadReader(payload.ReadBytes(payload.ReadInteger()))
                Dim pr As New PayloadReader(payload.ReadBytes(payload.ReadInteger()))
                'pixel.point = CType(bmpData.Scan0.ToPointer(), PointerToByte)
                pixel.point = New PointerToByte(bmpData.Scan0)
    
                Dim bit As New Bit()
    
                If BitsPr.Packet.Length > 0 Then
                    bit = New Bit(BitsPr.ReadByte())
                End If
                Dim BitIndex As Integer = 0
    
                While pr.Offset < pr.Packet.Length
                    Dim ConstValue As Byte = pr.ReadByte()
                    Dim count As Byte = pr.ReadByte()
                    Dim value As Byte = pr.ReadByte()
    
                    Dim direction__1 As Direction = Direction.Straight
                    If ConstValue > 0 Then
                        direction__1 = CType(bit.GetIndex(System.Math.Max(System.Threading.Interlocked.Increment(BitIndex), BitIndex - 1)), Direction)
    
                        If BitIndex = 8 Then
                            If BitsPr.Offset < BitsPr.Packet.Length Then
                                bit = New Bit(BitsPr.ReadByte())
                            End If
                            BitIndex = 0
                        End If
                    End If
    
                    For i As Integer = 0 To count - 1
                        Select Case target
                            Case TargetChannel.A
                                pixel.A = CByte(value)
                                Exit Select
                            Case TargetChannel.R
                                pixel.R = CByte(value)
                                Exit Select
                            Case TargetChannel.G
                                pixel.G = CByte(value)
                                Exit Select
                            Case TargetChannel.B
                                pixel.B = CByte(value)
                                Exit Select
                        End Select
    
                        Select Case direction__1
                            Case Direction.Down
                                value -= ConstValue
                                Exit Select
                            Case Direction.Up
                                value += ConstValue
                                Exit Select
                        End Select
                        pixel.NextPixel()
                    Next
                End While
                pr = Nothing
                target += 1
            Next
    
            payload = Nothing
            bitmap.UnlockBits(bmpData)
            Return bitmap
        End Function
    End Class


    • Edited by AndyjDel Sunday, February 3, 2013 3:43 PM
    Sunday, February 3, 2013 3:38 PM
  • If val > BeginValue Then direction__1 = Direction.Up ConstValue = CShort(val - BeginValue) ElseIf val < BeginValue Then direction__1 = Direction.Down MessageBox.Show(BeginValue) ConstValue = CShort(val - BeginValue) ElseIf val = BeginValue Then direction__1 = Direction.Straight ConstValue = 0 End If

    BeginValue is only 118

    I've uploaded my project at its current state if anyone could please take a look.

    http://fs07n5.sendspace.com/dl/d8aa7c1bc9e273cd3e583f945acf7edf/510e884c050bc2ea/sahhet/ImageCompression.zip

    • Edited by AndyjDel Sunday, February 3, 2013 3:55 PM
    Sunday, February 3, 2013 3:49 PM
  • "If you run following codes you can see the difference."

    That's ALWAYS code.

    Renee


    "MODERN PROGRAMMING is deficient in elementary ways BECAUSE of problems INTRODUCED by MODERN PROGRAMMING." Me


    Sunday, February 3, 2013 5:01 PM
  • I'd get familiar fast. Learn them.

    Renee


    "MODERN PROGRAMMING is deficient in elementary ways BECAUSE of problems INTRODUCED by MODERN PROGRAMMING." Me

    Sunday, February 3, 2013 5:08 PM
  • "If the output is exceeding  the limit CShort -32768 to 32767."

    This is true for a signed short. An unsigned short has a maximum value of 65534 or 2^16-1.

    Renee


    "MODERN PROGRAMMING is deficient in elementary ways BECAUSE of problems INTRODUCED by MODERN PROGRAMMING." Me

    Sunday, February 3, 2013 5:36 PM
  • In seems that the helper structure PointerToByte (discussed in other posts) needs an adjustment for the case of indexed access of bytes. Try this fixed Item property:

        Public Structure PointerToByte

            Default Public Property Item(ByVal i As Integer) As Byte

                Get

                    Return Marshal.ReadByte(mP + i)

                End Get

                Set(value As Byte)

                    Marshal.WriteByte(mP + i, value)

                End Set

            End Property

            . . . .

        End Structure

    Sunday, February 3, 2013 6:37 PM
  • You have constValue defined as a Byte, but you are trying to assign a Short value to it.

    HTH,

    Andrew

    Sunday, February 3, 2013 6:43 PM
  • "If the output is exceeding  the limit CShort -32768 to 32767."

    This is true for a signed short. An unsigned short has a maximum value of 65534 or 2^16-1.

    Renee


    "MODERN PROGRAMMING is deficient in elementary ways BECAUSE of problems INTRODUCED by MODERN PROGRAMMING." Me

    Hi Renee,

     I think you meant 65535.

    ;-)


    Regards,

    profile for John Anthony Oliver at Stack Overflow, Q&A for professional and enthusiast programmers

    Click this link to see the NEW way of how to insert a picture into a forum post.

    Installing VB6 on Windows 7

    App Hub for Windows Phone & XBOX 360 developers.

    Wednesday, February 6, 2013 11:44 AM