locked
Out-of-Memory error on Sort Transformation RRS feed

  • Question

  • Trying to determine if anyone else has encountered this problem and, if so, have you obtained a resolution:

    I have a data flow initiated from a Foreach Container. Inside the Data Flow there is a Source Script transformation connected to a Sort transformation with an OLE DB Destination.

    The Sort transformation has the sort keys defined and the "Eliminate Duplicates" option checked.

    The first pass into the Data Flow is successful. The 2nd pass fails in the Sort:

    Information: 0x4004300A at Load TC03, DTS.Pipeline: Validation phase is beginning.
    Information: 0x40043006 at Load TC03, DTS.Pipeline: Prepare for Execute phase is beginning.
    Information: 0x40043007 at Load TC03, DTS.Pipeline: Pre-Execute phase is beginning.
    Information: 0x4004300C at Load TC03, DTS.Pipeline: Execute phase is beginning.
    Warning: 0x80004005 at Load TC03, Sort [1506]: Unspecified error
    Error: 0xC020821B at Load TC03, Sort [1506]: A worker thread in the Sort transformation stopped with error code 0x80004005. A catastrophic error was encountered while sorting a buffer.
    Error: 0xC020821B at Load TC03, Sort [1506]: A worker thread in the Sort transformation stopped with error code 0x%1!8.8X!. A catastrophic error was encountered while sorting a buffer.
    Error: 0xC0208296 at Load TC03, Sort [1506]: The input buffer could not be cloned. An out-of-memory condition occurred or there was an internal error.
    Error: 0xC0047022 at Load TC03, DTS.Pipeline: The ProcessInput method on component "Sort" (1506) failed with error code 0x80004005. The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running.
    Error: 0xC0047021 at Load TC03, DTS.Pipeline: Thread "WorkThread0" has exited with error code 0x80004005.
    Error: 0xC0047039 at Load TC03, DTS.Pipeline: Thread "WorkThread1" received a shutdown signal and is terminating. The user requested a shutdown, or an error in another thread is causing the pipeline to shutdown.
    Error: 0xC0047021 at Load TC03, DTS.Pipeline: Thread "WorkThread1" has exited with error code 0xC0047039.
    Information: 0x40043008 at Load TC03, DTS.Pipeline: Post Execute phase is beginning.
    Information: 0x402090DF at Load TC03, hlp_ISOTypeLossTC03 [379]: The final commit for the data insertion has started.
    Information: 0x402090E0 at Load TC03, hlp_ISOTypeLossTC03 [379]: The final commit for the data insertion has ended.
    Information: 0x40043009 at Load TC03, DTS.Pipeline: Cleanup phase is beginning.
    Information: 0x4004300B at Load TC03, DTS.Pipeline: "component "hlp_ISOTypeLossTC03" (379)" wrote 5388 rows.
    Task failed: Load TC03
    Warning: 0x80019002 at Foreach TC03 File: The Execution method succeeded, but the number of errors raised (7) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
    Warning: 0x80019002 at ISOTypeLoss: The Execution method succeeded, but the number of errors raised (7) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
    SSIS package "ISOTypeLoss.dtsx" finished: Failure.


    Does this ring a bell with anyone?

    Thanks
    Wednesday, June 22, 2005 2:11 PM

Answers

  •  Raul Villaronga wrote:


    Any update as to whether the fix made the June CTP?


    It didn't. It'll be in the next CTP.
    Thursday, July 21, 2005 2:34 PM
  • Yes, this was caused by a bug in the sort transform. It was not correctly releasing some resources when the dataflow completed. The forloop just exposes the problem because the sort doesn't get freed between executions.
    Thursday, July 21, 2005 5:44 PM

All replies

  • I think this is a known issue that they are currently looknig into.
    its not necassarily a problem with the Sort, its more caused by the Foreach loop (I think).

    Check out thread "For Loop container - out of memory condition"

    -Jamie
    Wednesday, June 22, 2005 2:45 PM
  • Hi Raul,

    Yes, this is a known issue. We've fixed it for the next CTP. In the meantime, the only workaround I'm aware of is to put the data flow task in a separate package and use an Execute Package task to schedule the execution of the data flow task.
    regards,
    ash

    Wednesday, June 22, 2005 4:14 PM
  • What volume of data do you get this error on?

    It would be interesting to see if ExtraSort component (http://www.ivolva.com/extrasort.html) would hit the same problem.
    Wednesday, June 22, 2005 7:41 PM
  • Very small - about 300 rows
    Thursday, June 23, 2005 11:58 AM
  • Ash:

    I have the June CTP installed and am getting the same error:

    Warning: 0x80004005 at Load V6 Policy, Sort [4166]: Unspecified error

    Error: 0xC020821B at Load V6 Policy, Sort [4166]: A worker thread in the Sort transformation stopped with error code 0x80004005. A catastrophic error was encountered while sorting a buffer.

    Error: 0xC020821B at Load V6 Policy, Sort [4166]: A worker thread in the Sort transformation stopped with error code 0x%1!8.8X!. A catastrophic error was encountered while sorting a buffer.

    Error: 0xC0208296 at Load V6 Policy, Sort [4166]: The input buffer could not be cloned. An out-of-memory condition occurred or there was an internal error.

    Error: 0xC0047022 at Load V6 Policy, DTS.Pipeline: The ProcessInput method on component "Sort" (4166) failed with error code 0x80004005. The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running.

    Error: 0xC0047021 at Load V6 Policy, DTS.Pipeline: Thread "WorkThread0" has exited with error code 0x80004005.

    Error: 0xC0047039 at Load V6 Policy, DTS.Pipeline: Thread "WorkThread1" received a shutdown signal and is terminating. The user requested a shutdown, or an error in another thread is causing the pipeline to shutdown.

    Error: 0xC0047021 at Load V6 Policy, DTS.Pipeline: Thread "WorkThread1" has exited with error code 0xC0047039.


    Another poster suggested a thread for "For Loop container - out of memory condition" - I searched this forum and it returned no results.

    Any update as to whether the fix made the June CTP?

    Thursday, July 21, 2005 2:00 PM
  •  Raul Villaronga wrote:


    Any update as to whether the fix made the June CTP?


    It didn't. It'll be in the next CTP.
    Thursday, July 21, 2005 2:34 PM
  • Yes, this was caused by a bug in the sort transform. It was not correctly releasing some resources when the dataflow completed. The forloop just exposes the problem because the sort doesn't get freed between executions.
    Thursday, July 21, 2005 5:44 PM
  • FYI - Still getting this in September CTP.

    Sunday, September 18, 2005 10:47 PM
  • Both memory issues I had with June CTP including the catastrophic failure of the sort in a loop and the data viewer consuming memory have been fixed in September CTP.
    Monday, September 19, 2005 10:30 PM
  • Was this really fixed in 2005 September CTP?

    I recently created a package in SSIS 2005 that does multiple sorts with in a data flow.  It uses peak memory of 500MB.  I have started getting the same error on a particular sort transformation.

    Wednesday, October 12, 2011 6:37 PM