Overflow the disk I/O buffer

Question

Hello,

I am getting "overflow the disk I/O buffer" in my SSIS, and what's weird is that when I construct the same SSIS in a new package, it works perfectly.  I almost want to believe that it could be a bug.  Some days when I import the files, it works fine, but some days it errors out with this error on the last column.  Is there some setting with CR/LF or LF that I have to pay attention to avoid this type of random error?

 

Thanks for your help!

-Lawrence

Answer 1

Can you please provide the exact error message and any other error messages around it?

Thanks,
Phil

Answer 2

[Flat File Source [439]] Error: The column data for column "Avg Position" overflowed the disk I/O buffer.

[Flat File Source [439]] Error: An error occurred while skipping data rows.

[DTS.Pipeline] Error: The PrimeOutput method on component "Flat File Source" (439) returned error code 0xC0202091.  The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing.

[DTS.Pipeline] Error: Thread "SourceThread0" has exited with error code 0xC0047038.

[DTS.Pipeline] Error: Thread "WorkThread0" received a shutdown signal and is terminating. The user requested a shutdown, or an error in another thread is causing the pipeline to shutdown.

[DTS.Pipeline] Error: Thread "WorkThread0" has exited with error code 0xC0047039.

Answer 3

What is the data type for the column, "Avg Position" and how many bytes is each row in the file?

Answer 4

Has there been a resolution to this? I have a similar problem occurring.

Answer 5

Joshua Kasper wrote:

Has there been a resolution to this? I have a similar problem occurring.

No, because SQLChamp needs to respond to my last questions.

Answer 6

Column in database is float.

Bytes from the text file range from 1 to 35.6

-Lawrence 

 

Answer 7

Joshua Kasper wrote:

Has there been a resolution to this? I have a similar problem occurring.

And do you have a similar problem, or the exact same problem?  Do you have the same error messages?

Answer 8

SQLChamp wrote:

Column in database is float. Bytes from the text file range from 1 to 35.6 -Lawrence

What is the data type of the column inside SSIS?  Make sure that it is what you expect it to be in all stages of the data flow.

Answer 9

I doubled check the Data Flow, and they are all float or mapping to the same column name.

Maybe the format of the data in the text file is causing SSIS to fail?  Would you know what kind of data in the text file would do that?  But I would assume the error log should indicate which row it error out on if it were format issue. 

 

Any other places to check?  or would this be random SSIS bug?

-Lawrence

Answer 10

SQLChamp wrote:

I doubled check the Data Flow, and they are all float or mapping to the same column name. Maybe the format of the data in the text file is causing SSIS to fail?  Would you know what kind of data in the text file would do that?  But I would assume the error log should indicate which row it error out on if it were format issue.    Any other places to check?  or would this be random SSIS bug? -Lawrence

Well, there isn't a float data type inside a data flow...  Can you tell me what SSIS data type you have this column listed as?

Also, if you have a numeric field defined as numeric(4,2) and you have a number such as 1.234, it will fail, strictly speaking.  12.34 works.  123.4 does not.

Answer 11

FlatFile Metadata for [Avg Position] = DT_R4 (precision=0, length=0)

Float cannot set precision like numeric, so hopefuly precision is not the cause of this SSIS error.

What's weird is that ther are many other columns with Float, and they don't appear to cause any error, but [Avg Position] is at the last column, so not sure if that has an effect.

 

I am trying to attach the screenshot in this forum, but I am not sure you are able to see it below.  Maybe this forum don't support screenshot.

-Lawrence

 

 

 

 

Answer 12

Since it's the last field in the file, it could be a row terminator issue.

How do you have the flat file setup?  Ragged right?  CSV? 

Answer 13

In SSIS, [Avg Position] ColumnDelimiter is set to {CR}-{LF}

Row Delimiter - {CR}-{LF}

How do I check whether my text file source is compatible with the setting above?

 

-Lawrence

Answer 14

I think the issue is that there is an extra blank line at the end.  This does not happen all the time, so have to accommodate both (end of line & extra line).  How would I setup SSIS to accommodate the extra blank line? 

 

Thanks,

-Lawrence

Answer 15

Locale was set non-Unicode.

I wonder if setting it to Unicode 1252 (Latin 1) would resolve the issue, even though all the column datatype I have set does not have any unicode.

How does SSIS process the end of file (extra blank line VS end of file) compared between (unicode and no unicode)?

 

-Lawrence

 

Answer 16

Hello,

Can anyone offer any tips to resolve extra blank lines at end of the text file?

I am still running into issues whenever there are extra blank lines.  Is there a way to accomodate this?

I have to switch back and forth between Unicode setting in the automation.  

 

Appreciated,

-Lawrence 

Answer 17

Is there anyone who solved this issue? I have the same problem with my csv files...

 

 

Answer 18

In your File Connection Manager try to set "Header row delimiter" to {LF} in both General and Columns tab.

Answer 19

Could you pre-process your file to remove extra blank lines?

 

Thanks,

Bob

 

Answer 20

Hi,

Just stumbled across this forum as I was having the same problem. Don't know if this will help, but my issue came down to the fact that the encoding of the file I was importing had changed from ANSI to Unicode. I changed it back to ANSI and it seemed to work fine.

Hope this helps,

Leo