none
How to load a large text file to RichTextBox? RRS feed

  • Question

  • I want to load a 410 MB text file to RichTextBox but the following methods did not work fine.
    Please give me an advice and it will be very helpful if you can paste the sample code for understanding.

    1. The following codes cause OutofMemoryException
    System.IO.StreamReader textFile = new System.IO.StreamReader(@"c:\data.txt"); 
    richTextBox.Text = textFile.ReadToEnd();
    textFile.Close();

    2. The following codes also cause OutofMemoryException
    richTextBox.LoadFile(@"c:\data.txt", RichTextBoxStreamType.PlainText);

    3. The following codes don't cause OutofMemoryException but load the file too slowly
    StreamReader sr = new StreamReader(@"c:\data.txt");
    while (!sr.EndOfStream)
    {
        richTextBox.Text += "\n" + sr.ReadLine();
    }

    Thanks and regards.
    Land Ocean
    • Moved by CoolDadTx Thursday, January 21, 2010 2:17 PM WinForms related (From:Visual C# General)
    Wednesday, January 20, 2010 4:43 PM

Answers

  • Have you cosidered the possibility that the amount of memory available to your application may be limited?  512 MBytes.
    Mark the best replies as answers. "Fooling computers since 1971."
    • Marked as answer by Land Ocean Saturday, January 23, 2010 5:08 PM
    Saturday, January 23, 2010 2:42 PM
  • Have you cosidered the possibility that the amount of memory available to your application may be limited?  512 MBytes.
    Mark the best replies as answers. "Fooling computers since 1971."

    .NET wants contiguous memory for most objects.  1/2 GB is about the limit on 32 bit OS.
    • Marked as answer by Land Ocean Saturday, January 23, 2010 5:07 PM
    Saturday, January 23, 2010 3:23 PM

All replies

  • Trying to place a 410MB text file into a RichTextBox is going to use a lot of memory, any way you attempt it.  410MB text files, if the text is in ascii encoding, is going to be 810MB once you read it in (since strings in C# are unicode, ie 2bytes/char).

    Your first two approachs are probably running out of memory because you're reading the entire file into memory, holding it there, then assigning it to your RichTextBox.  This causes two copies to be loaded.

    The third approach avoids this, but at the cost of constantly "thrashing" your memory.  You're going to be causing the GC to fire constantly, since you're creating new strings (RichTextBox.Text) in a loop.

    That being said - my first suggestion: Don't try to do this at all.  A 410MB file is too much text to be very useful in a RichTextBox.  I would recommend trying to just read in the portion that is important, and display just that - not all 410MB...

    If you do need to do this, instead of reading the file, and assigning the text, you should use RichTextBox.LoadFile - this allows the RichTextBox to directly load the file from disk (or your stream), which will most likely help keep the memory consumption down.


    Reed Copsey, Jr. - http://reedcopsey.com
    Wednesday, January 20, 2010 5:00 PM
    Moderator
  • Thanks Reed

    I have tried RichTextBox.LoadFile but the OutOfMemoryException still occurred.
    I agree with your suggestion to load only 1 page of text data to RichTextBox and use the scrollbar click event handler to load the next or previous page. But I don't know how to make it. Do you have a sample or any link of this solution?

    BTW, when I used RichTextBox.LoadFile, the OutOfMemoryException occurred but the 410 MB text file was completely loaded to RichTextBox. The max line number is over 3,900,000.

    Thanks and regards.
    Land Ocean
    Thursday, January 21, 2010 2:01 PM
  • Try using a different type of Stream to read the file.
    Try reading your file into a BufferedStream in memory. 
    You will not have random access, but you will still be able to scroll through it from start to finish.

    BufferedStream Class (System.IO)

    Note that you can specify the size of the buffer in the constructor.

    Rudy  =8^D

    Mark the best replies as answers. "Fooling computers since 1971."
    Thursday, January 21, 2010 2:15 PM
  • I tried to use BufferedStream but the the OutOfMemoryException occurred again.
    My code is:

    =========================
    System.IO.FileInfo fi = new System.IO.FileInfo(ofdMain.FileName);
    System.IO.FileStream fs = new System.IO.FileStream(ofdMain.FileName, System.IO.FileMode.Open);
    System.IO.BufferedStream bs = new System.IO.BufferedStream(fs);
    int bufferSize = 10000000;      //Set the buffer size to 10MB.
    //int bufferSize = 100000000;    //100MB looks to big as buffer size.
    Byte[] buffer = new byte[bufferSize];
    int readLength;
    do
    {
        readLength = bs.Read(buffer, 0, bufferSize);
        newRich.Text += System.Text.Encoding.ASCII.GetString(buffer);
        Application.DoEvents();
    } while (readLength == bufferSize);
    =========================

    I found that over 50MB text data were properly loaded to RichTextBox before the exception appeared.
    I am doubting if there is a memory leak but don't know where is wrong.
    Could you have a check?

    Thanks and regards.


    Land Ocean
    Friday, January 22, 2010 2:23 PM


  • BufferedStream Class (System.IO)

    Note that you can specify the size of the buffer in the constructor.


    Use the constructor for the BufferedStream that specifies a maximum size.

    Mark the best replies as answers. "Fooling computers since 1971."
    Friday, January 22, 2010 2:35 PM
  • I changed following codes.

    ===========================
    System.IO.FileStream fs = new System.IO.FileStream(ofdMain.FileName, System.IO.FileMode.Open);
    System.IO.BufferedStream bs = new System.IO.BufferedStream(fs, (int)fi.Length);
    ===========================

    Is it you mentioned?
    If it was wrong, could you give me a sample code?

    Thanks !
    Land Ocean
    Friday, January 22, 2010 2:59 PM
  • Sample code is at the link that I provided.  I think you need to click "Constructors", and then click on a specific overload to get details.  Did you look at the value of "fi.Length"?  I would have expected you to use the obvious choice of your variables, "bufferSize".


    Mark the best replies as answers. "Fooling computers since 1971."
    Friday, January 22, 2010 3:18 PM
  • fi.Length = 422507347 byte
    It is exactly the 410 MB file's size.
    I tried following code but still couldn't avoid the OutOfMemoryException error.

    =====================================
    System.IO.FileInfo fi = new System.IO.FileInfo(@"c:\data.txt");
    System.IO.FileStream fs = new System.IO.FileStream(@"c:\data.txt", System.IO.FileMode.Open);
    int bufferSize = 10000000;    //Set the buffer size to 10MB.

    using (System.IO.BufferedStream bs = new System.IO.BufferedStream(fs, bufferSize))
    {
        Byte[] buffer = new byte[bufferSize];
        int readLength;

        do
        {
            readLength = bs.Read(buffer, 0, bufferSize);
            newRich.Text += System.Text.Encoding.ASCII.GetString(buffer);
            Application.DoEvents();
        } while (readLength == bufferSize);

        bs.Close();
    }

    fs.Close();
    =====================================

    Please tell me what caused the OutOfMemoryException error.

    Thanks and regards.


    Land Ocean
    Friday, January 22, 2010 4:58 PM
  • fi.Length = 422507347 byte
    It is exactly the 410 MB file's size.
    I tried following code but still couldn't avoid the OutOfMemoryException error.

    =====================================
    System.IO.FileInfo fi = new System.IO.FileInfo(@"c:\data.txt");
    System.IO.FileStream fs = new System.IO.FileStream(@"c:\data.txt", System.IO.FileMode.Open);
    int bufferSize = 10000000;    //Set the buffer size to 10MB.

    using (System.IO.BufferedStream bs = new System.IO.BufferedStream(fs, bufferSize))
    {
        Byte[] buffer = new byte[bufferSize];
        int readLength;

        do
        {
            readLength = bs.Read(buffer, 0, bufferSize);
            newRich.Text += System.Text.Encoding.ASCII.GetString(buffer);
            Application.DoEvents();
        } while (readLength == bufferSize);

        bs.Close();
    }

    fs.Close();
    =====================================

    Please tell me what caused the OutOfMemoryException error.

    Thanks and regards.


    Land Ocean


    Why are you even asking me why did it occur?
    You know where it occurred, on that bold line.
    You should know why it occurred, too.  Stubbornness.

    You must understand this.  You just simply cannot display all of the text at once.
    Give it up.  You have been told that the file is too large already. 

    That bold line of code adds more and more text to the control until you run out of memory.
    Either you give it up trying, or I will.

    You must decide how much of the file you wish to display at one time.
    We cannot tell you how to do that, or even suggest a way to do it. 
    WHY?  We don't have a single clue as to exactly what is in your file, much less the format of it.

    Hope this helps.

    Rudy  =8^D


    Mark the best replies as answers. "Fooling computers since 1971."
    • Edited by Rudedog2 Saturday, January 23, 2010 3:33 PM
    Friday, January 22, 2010 5:43 PM
  • When I create a RichTextBox control, the MaxLength = 2147483647.
    I think RichTextBox can opne a file less than 2GB. That's why I did not give up.
    Sorry for my stubbornness, I just want to know the truth.
    If it is a RichTextBox's restriction or issue, of cause I will give up. But if it is my coding problem, I still hope someone can help to correct it.
    Regarding the text file's format, it is just a plane text and it is a log file of DebugView tool. Nothing special.

    Thanks and regards.
    Land Ocean
    Saturday, January 23, 2010 2:58 AM
  • Have you cosidered the possibility that the amount of memory available to your application may be limited?  512 MBytes.
    Mark the best replies as answers. "Fooling computers since 1971."
    • Marked as answer by Land Ocean Saturday, January 23, 2010 5:08 PM
    Saturday, January 23, 2010 2:42 PM
  • Have you cosidered the possibility that the amount of memory available to your application may be limited?  512 MBytes.
    Mark the best replies as answers. "Fooling computers since 1971."

    .NET wants contiguous memory for most objects.  1/2 GB is about the limit on 32 bit OS.
    • Marked as answer by Land Ocean Saturday, January 23, 2010 5:07 PM
    Saturday, January 23, 2010 3:23 PM
  • Thanks guys!

    I understand the RichTextBox object should have a limit of memory.
    I have given up loading 410 MB to RichTextBox because it is not a smart choice.
    I am thinking about the following algorithm:

    1. Calculate the amount of bytes in 1 page of RichTextBox.
    2. Append 1 pages data to RichTextBox.
    3. Chage the data to next or previous page when user click the scrollbar.

    I don't have confidence to make it because there are many difficult point for me.
    But I'd like to try and report the result to you after it comes true.

    Thanks for your helps and suggestions again. 
    Land Ocean
    Saturday, January 23, 2010 5:36 PM