none
關於 大量多次 WriteLine()的問題 RRS feed

  • 問題

  • 請問各位先進,
    當我處理大量文字輸出到檔案時(CSV)
    使用StreamWriter的WriteLine指令來輸出

    要輸出的資料大約有5百萬行 (檔案大小約4GB)
    當寫到一半 即2GB時 會出現以下錯誤訊息

    {"Must complete Convert() operation or call Encoder.Reset() before calling GetBytes() or GetByteCount(). Encoder 'Chinese Traditional (Big5)' fallback 'System.Text.InternalEncoderBestFitFallback'."}

    StackTrace::
      at System.Text.DBCSCodePageEncoding.GetBytes(Char* chars, Int32 charCount, Byte* bytes, Int32 byteCount, EncoderNLS encoder)
       at System.Text.EncoderNLS.GetBytes(Char* chars, Int32 charCount, Byte* bytes, Int32 byteCount, Boolean flush)
       at System.Text.EncoderNLS.GetBytes(Char[] chars, Int32 charIndex, Int32 charCount, Byte[] bytes, Int32 byteIndex, Boolean flush)
       at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
       at System.IO.StreamWriter.Write(Char[] buffer, Int32 index, Int32 count)
       at System.IO.TextWriter.WriteLine(String value)
    at ....

    程式如下:


    FileStream aFileStream = null;
    StreamWriter aStreamWriter = null;
    aFileStream = new FileStream(strFilePath, FileMode.Create, FileAccess.ReadWrite);

    aStreamWriter = new StreamWriter(aFileStream, Encoding.GetEncoding(950));

    for (int iRowIndex = 0; iRowIndex < oExportSourceData_.RowCount; iRowIndex++)
    {
    StringBuilder oOneRowOfExportData = new StringBuilder();

    oOneRowOfExportData.Append(oExportSourceData_[iRowIndex, iSelectedColumnIndex].ToString() + strDelimiter);

    aStreamWriter.WriteLine(oOneRowOfExportData.ToString());

    }


    搜尋一下Google, 似乎encoding為Big5 和日文 JIS的會出現這樣的問題

    請問是不是StreamWriter WriteLine的限制
    有沒有什麼辦法可以解呢?

    謝謝各位先進



    2009年1月16日 上午 02:57

解答

  •  

    資料太大了   改成分批寫入  看看

     

    Code Snippet

     

     

    FileStream aFileStream = null;
    StreamWriter aStreamWriter = null;
    aFileStream = new FileStream(strFilePath, FileMode.Create, FileAccess.ReadWrite);


     

     
    for (int iRowIndex = 0; iRowIndex < oExportSourceData_.RowCount; iRowIndex++)
    {
    StringBuilder oOneRowOfExportData = new StringBuilder();

    oOneRowOfExportData.Append(oExportSourceData_[iRowIndex, iSelectedColumnIndex].ToString() + strDelimiter);

     

       aStreamWriter   = new StreamWriter(aFileStream, Encoding.GetEncoding(950));


       aStreamWriter.WriteLine(oOneRowOfExportData.ToString());

     

       aStreamWriter.Close();


    }

     

     

     

    2009年1月16日 上午 04:29