none
long integers division close to whole number rounds up instead of down with Math.Floor() RRS feed

  • Question

  • I have a filing system that divides my massive file into a series of smaller files, opens the FileStream and positions it at a location relative to the intIndex_Revised(times record size).  All records are indexed and all works well.  But at a large number like the one below, where the value inside the Math.Floor() should be 33.9999999 its output is 34 which then gives me an inIndex_Revised (for the 35th file stream counting zero) of -1 (times intRecordSize) which crashes my system.

    public static int intCW_LL_NumRecPerFile = (int)Math.Pow(2, 19);

    public static classFileStreamManager CW_LL_Rec_FileStream_Get(long index)

    index = 17825791; classFileStreamManager cFS = null; int intFileNumber = (int)Math.Floor( (float) ((double)index / (double)intCW_LL_NumRecPerFile)); int intIndex_Revised = (int)((long)index - (long)intFileNumber * (long)intCW_LL_NumRecPerFile); ...

    }

    the division inside the Math.Floor() was originally

    int intFileNumber = (int)Math.Floor((float)index/(float)intCW_LL_NumRecPerFile));

    but I changed it when i ran was watching it through the Debugger to catch where it crashed.  It doesn't fix the problem so I added the following lines to catch and correct it.

    if (intIndex_Revised < 0)
    {
        intFileNumber--;
        intIndex_Revised = (int)((long)index - (long)intFileNumber * (long)intCW_LL_NumRecPerFile);
        if (intIndex_Revised > intCW_LL_NumRecPerFile)
        {
            System.Windows.Forms.MessageBox.Show("Error calculating CW__LL_REc_Filestream_Gret()");
        }
    }

    This works(for now) but is there a better way?

    BadButBit


    my code is perfect until i don't find a bug


    Sunday, March 17, 2019 9:25 AM

Answers

  • Check this:


    int intFileNumber = (int)(index / intCW_LL_NumRecPerFile);

    int intIndex_Revised = (int)(index - intFileNumber * intCW_LL_NumRecPerFile);


    • Edited by Viorel_MVP Sunday, March 17, 2019 9:55 AM
    • Marked as answer by Christ Kennedy Sunday, March 17, 2019 11:34 AM
    Sunday, March 17, 2019 9:55 AM

All replies

  • Check this:


    int intFileNumber = (int)(index / intCW_LL_NumRecPerFile);

    int intIndex_Revised = (int)(index - intFileNumber * intCW_LL_NumRecPerFile);


    • Edited by Viorel_MVP Sunday, March 17, 2019 9:55 AM
    • Marked as answer by Christ Kennedy Sunday, March 17, 2019 11:34 AM
    Sunday, March 17, 2019 9:55 AM
  • nice.

    so, lose the Math.Floor() and the integers round down anyway?

    BadButBit


    my code is perfect until i don't find a bug

    Sunday, March 17, 2019 11:34 AM