none
SSIS ERROR : overflowed the disk I/O buffer, DTS_E_PRIMEOUTPUTFAILED RRS feed

  • Question

  • Hi,
    I get folllwing error while SSIS package is executing and uploading Data from Flat Files to SQL Server 2005. This Error goes away when I change my SSIS Package Connection Manager to read UNICODE data files.
    Is there any smart way to figure out which flat files have UniCode Data  in the and which is not a Unicode data file.

    Thanks,
    Vinod


    Information: 0x402090DC at Upload EP Data, DAT File Reader [1]: The processing of file "C:\Data2\EP05PF2000002_070412_002921.dat" has started.
    Information: 0x4004300C at Upload EP Data, DTS.Pipeline: Execute phase is beginning.
    Error: 0xC020209C at Upload EP Data, DAT File Reader [1]: The column data for column "SYSTEM_LOCKED" overflowed the disk I/O buffer.
    Error: 0xC0202091 at Upload EP Data, DAT File Reader [1]: An error occurred while skipping data rows.
    Error: 0xC0047038 at Upload EP Data, DTS.Pipeline: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED.  The PrimeOutput method on component "DAT File Reader" (1) returned error code 0xC0202091.  The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing.  There may be error messages posted before this with more information about the failure.
    Error: 0xC0047021 at Upload EP Data, DTS.Pipeline: SSIS Error Code DTS_E_THREADFAILED.  Thread "SourceThread0" has exited with error code 0xC0047038.  There may be error messages posted before this with more information on why the thread has exited.
    Error: 0xC0047039 at Upload EP Data, DTS.Pipeline: SSIS Error Code DTS_E_THREADCANCELLED.  Thread "WorkThread0" received a shutdown signal and is terminating. The user requested a shutdown, or an error in another thread is causing the pipeline to shutdown.  There may be error messages posted before this with more information on why the thread was cancelled.
    Error: 0xC0047021 at Upload EP Data, DTS.Pipeline: SSIS Error Code DTS_E_THREADFAILED.  Thread "WorkThread0" has exited with error code 0xC0047039.  There may be error messages posted before this with more information on why the thread has exited.
    Information: 0x40043008 at Upload EP Data, DTS.Pipeline: Post Execute phase is beginning.
    Information: 0x402090DD at Upload EP Data, DAT File Reader [1]: The processing of file "C:\Data2\EP05PF2000002_070412_002921.dat" has ended.
    Thursday, November 1, 2007 7:05 PM

Answers

  • If you are still looking at this, I converted the C# code to a script task, and put together a sample package that checks to see if the file is ASCII or Unicode, and executes the appropriate data flow depending on the results.

     

    http://agilebi.com/cs/blogs/jwelch/archive/2007/11/08/dealing-with-unicode-and-ascii-text-files.aspx

     

    It's a bit more involved than I originally thought, because you can't just set the Unicode property (it blows up the pipeline because the metadata is different). Instead, you need to have two data flows (one for ASCII and one for Unicode) and conditionally choose which one to execute based on the results of the script task.
    Thursday, November 8, 2007 10:24 PM
    Moderator

All replies

  •  vinod kushwaha wrote:
    Hi,
    Is there any smart way to figure out which flat files have UniCode Data  in the and which is not a Unicode data file.

     

    I'm honestly not sure what you're asking here. Do you want to know how to tell whether a given flat file is Unicode or ASCII, or are you asking something different?

    Friday, November 2, 2007 2:32 AM
    Moderator
  • Yes, How do I detect, given flat file is UNICODE or not? So that I can pass it to a correct Connection Manager.

     

    Thanks,

    VInod

     

    Friday, November 2, 2007 2:45 PM
  • That's an excellent question, and my (admittedly brief) research has not turned up an answer.

     

    Since the first 128 characters of Unicode and ASCII encodings are the same, will it work if you tell SSIS that all of your files are Unicode?

     

    Friday, November 2, 2007 8:46 PM
    Moderator
  • I get following warning when I process Non-Unicode files with Unicode Flat File Connection Manager.

    Thanks,
    Vinod


    Information: 0x402090DC at Data Flow Task, Flat File Source [1]: The processing of file "C:\Data2\EP02PF2000017_070531_222912_2.dat" has started.
    Information: 0x4004300C at Data Flow Task, DTS.Pipeline: Execute phase is beginning.
    Warning: 0x8020202B at Data Flow Task, Flat File Source [1]: The end of the data file was reached while reading header rows. Make sure the header row delimiter and the number of header rows to skip are correct.
    Information: 0x402090DE at Data Flow Task, Flat File Source [1]: The total number of data rows processed for file "C:\Data2\EP02PF2000017_070531_222912_2.dat" is 0.
    Information: 0x402090DF at Data Flow Task, OLE DB Destination [546]: The final commit for the data insertion has started.
    Information: 0x402090E0 at Data Flow Task, OLE DB Destination [546]: The final commit for the data insertion has ended.
    Information: 0x40043008 at Data Flow Task, DTS.Pipeline: Post Execute phase is beginning.
    Information: 0x402090DD at Data Flow Task, Flat File Source [1]: The processing of file "C:\Data2\EP02PF2000017_070531_222912_2.dat" has ended.
    Information: 0x40043009 at Data Flow Task, DTS.Pipeline: Cleanup phase is beginning.
    Information: 0x4004300B at Data Flow Task, DTS.Pipeline: "component "OLE DB Destination" (546)" wrote 0 rows.
    Monday, November 5, 2007 3:26 PM
  •  vinod kushwaha wrote:
    I get following warning when I process Non-Unicode files with Unicode Flat File Connection Manager.

    Thanks,
    Vinod


    Information: 0x402090DC at Data Flow Task, Flat File Source [1]: The processing of file "C:\Data2\EP02PF2000017_070531_222912_2.dat" has started.
    Information: 0x4004300C at Data Flow Task, DTS.Pipeline: Execute phase is beginning.
    Warning: 0x8020202B at Data Flow Task, Flat File Source [1]: The end of the data file was reached while reading header rows. Make sure the header row delimiter and the number of header rows to skip are correct.
    Information: 0x402090DE at Data Flow Task, Flat File Source [1]: The total number of data rows processed for file "C:\Data2\EP02PF2000017_070531_222912_2.dat" is 0.
    Information: 0x402090DF at Data Flow Task, OLE DB Destination [546]: The final commit for the data insertion has started.
    Information: 0x402090E0 at Data Flow Task, OLE DB Destination [546]: The final commit for the data insertion has ended.
    Information: 0x40043008 at Data Flow Task, DTS.Pipeline: Post Execute phase is beginning.
    Information: 0x402090DD at Data Flow Task, Flat File Source [1]: The processing of file "C:\Data2\EP02PF2000017_070531_222912_2.dat" has ended.
    Information: 0x40043009 at Data Flow Task, DTS.Pipeline: Cleanup phase is beginning.
    Information: 0x4004300B at Data Flow Task, DTS.Pipeline: "component "OLE DB Destination" (546)" wrote 0 rows.


    This sounds like a row delimiter issue, to me.
    Monday, November 5, 2007 3:32 PM
    Moderator
  • Hi Phil,
    This Warnig is not seen when I create my Flat File Connection manager using this Specific data file. But Then It would fail for some data files which have UNICODE data with same data header, giving errors displayed in my first post. How do I detect which files are UNICODE or NOT?

    Thanks,
    Vinod

    Monday, November 5, 2007 6:14 PM
  • Thanks for your Tip, I don't have VB skills, I think scripting task is the solution for this.
    Vinod

    Tuesday, November 6, 2007 2:50 PM
  • If you are still looking at this, I converted the C# code to a script task, and put together a sample package that checks to see if the file is ASCII or Unicode, and executes the appropriate data flow depending on the results.

     

    http://agilebi.com/cs/blogs/jwelch/archive/2007/11/08/dealing-with-unicode-and-ascii-text-files.aspx

     

    It's a bit more involved than I originally thought, because you can't just set the Unicode property (it blows up the pipeline because the metadata is different). Instead, you need to have two data flows (one for ASCII and one for Unicode) and conditionally choose which one to execute based on the results of the script task.
    Thursday, November 8, 2007 10:24 PM
    Moderator
  • Thanks a Lot . Your code in blog post is God send for me.

    Cheers,

    Vinod

    Friday, November 9, 2007 12:07 AM
  • Hi vinod,

    i am trying to load data from fastlane to sql server2005 data base using SSIS Package, i am getting following error. please tell me solution for this issue.

     

     Code: 0xC0047038
       Source: Insert SoldFL6 DTS.Pipeline
       Description: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED.  The PrimeOutput method on component "DataReader Source" (1) returned error code 0xC02090F5.  The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing.  There may be error messages posted before this with more information about the failure.
    End Error
    Error: 2008-09-28 07:51:26.34
       Code: 0xC0047021
       Source: Insert SoldFL6 DTS.Pipeline
       Description: SSIS Error Code DTS_E_THREADFAILED.  Thread "SourceThread0" has exited with error code 0xC0047038.  There may be error messages posted before this with more information on why the thread has exited.
    End Error
    Error: 2008-09-28 07:51:26.34
       Code: 0xC0047039
       Source: Insert SoldFL6 DTS.Pipeline
       Description: SSIS Error Code DTS_E_THREADCANCELLED.  Thread "WorkThread0" received a shutdown signal and is terminating. The user requested a shutdown, or an error in another thread is causing the pipeline to shutdown.  There may be error messages posted before this with more information on why the thread was cancelled.
    End Error
    Error: 2008-09-28 07:51:26.34
       Code: 0xC0047021
       Source: Insert SoldFL6 DTS.Pipeline
       Description: SSIS Error Code DTS_E_THREADFAILED.  Thread "WorkThread0" has exited with error code 0xC0047039.  There may be error messages posted before this with more information on why the thread has exited.
    End Error

     

    Monday, September 29, 2008 6:38 AM
  •  

    Hi,

     

    these elements are not available using EWS. If you really need access to those items, you need to revert to another access method (MAPI, WebDAV, ...).

     

    Kind regards,

    Henning Krause

    Wednesday, November 5, 2008 5:00 PM
  • I ran into this "overflowed the disk I/O buffer" error with a flat file as well and narrowed it down to switching the other way from unicode to ANSI.  Trying different sizes, the extract succeeded until my file exceeded 128k in size (131072 bytes) - you might want to check for that if you've happened across this thread.

    Sunday, December 9, 2012 8:36 PM