none
What is the fast way to write data to file and read them back? RRS feed

  • Question

  • Hello,everyone.

    Now I have a very big problem: I need to write about ten millions of data to a file and then read them back. So, i would very much like to know which is the fast way to do so. Now, I use the BinaryReader way.

    private bool SaveLocalData(List<Record> recordList)
            {
                try
                {
                    FileStream fileStream = new FileStream(FileName, FileMode.Create);
                    BinaryWriter writer = new BinaryWriter(fileStream);
                    writer.Write(recordList.Count);
    
                    for (int i = 0; i < recordList.Count; i++)
                    {
                        //writer.Write(recordList[i].ID);                    
                    }
    
                    writer.Close();
                    fileStream.Close();
                }
                catch (System.Exception e)
                {
                    MessageBox.Show(e.ToString());
                }          
    
                return true;
            }
    How to?

    Thursday, September 24, 2015 1:57 AM

Answers

  • One of the ways is using an intermediate buffer:

    using( Stream fileStream = new FileStream( FileName, FileMode.Create ),
           bs = new BufferedStream( fileStream ) )
    {
        using( var writer = new BinaryWriter( bs ) )
        {
            writer.Write( recordList.Count );
            . . .
        }
    }


    • Edited by Viorel_MVP Thursday, September 24, 2015 7:09 AM
    • Marked as answer by waterharbin Sunday, September 27, 2015 4:10 AM
    Thursday, September 24, 2015 7:08 AM

All replies

  • Hi waterharbin,

    ->What is the fast way to write data to file and read them back?

    I suggest you use BufferedStream to enhance the efficiency of reading and writing. A buffer is a block of bytes in memory used to cache data, thereby reducing the number of calls to the operating system. Buffers improve read and write performance. A buffer can be used for either reading or writing, but never both simultaneously. The Read and Write methods of BufferedStream automatically maintain the buffer.

    For more informations and examples, link below is for your reference.
    https://msdn.microsoft.com/en-us/library/system.io.bufferedstream(v=vs.110).aspx

    In addition, if the data are huge and structured. I suggest you use Local DataBase instead of file to store data. Follow below article to use Local DataBase.
    http://www.codeproject.com/Tips/811432/Use-Local-DataBase-file-sdf-in-Windows-Application

    Best Regards,
    Li Wang

    Thursday, September 24, 2015 7:08 AM
    Moderator
  • One of the ways is using an intermediate buffer:

    using( Stream fileStream = new FileStream( FileName, FileMode.Create ),
           bs = new BufferedStream( fileStream ) )
    {
        using( var writer = new BinaryWriter( bs ) )
        {
            writer.Write( recordList.Count );
            . . .
        }
    }


    • Edited by Viorel_MVP Thursday, September 24, 2015 7:09 AM
    • Marked as answer by waterharbin Sunday, September 27, 2015 4:10 AM
    Thursday, September 24, 2015 7:08 AM