none
Thread Pool Exhaustion - Polling with Oracle Adapter in BizTalk RRS feed

  • Question

  • Hi, All.

     

    I have a quite large problem with the Oracle Adapter from the Adapter Pack. In BizTalk I have a few receive locations that poll an Oracle table and deletes the rows on PostPoll. When the receive locations are enabled, BizTalk begins to "loose" worker threads in its thread pool, over time resulting in a situation where the BizTalk host isn't able to process anything due to a lack of worker threads.

     

    I haven't been able to determine the cause of the problem, neither have I found an explanation of what I'm seeing anywhere on the internet. Do any of you have an idea whats is happening (or perhaps just a workaround)?

     

    The whole setup is pretty standard, with no custom components involved. Most of the settings on the receive location are the standard settings from the wizard.

     

    Any input on the issue will be very welcome!

    Yours sincerely, Mads.

     

     

    Symptoms:

    • When the receive locations using the Oracle Adapter are disabled, the problem doesn't occur
    • The problem doesn't occur for send ports
    • When one/more of the receive locations are enabled, the thread count for the BizTalk process slowly rises
    • If the receive locations are disabled, the thread count quickly falls to the normal level
    • Low values for receiveTimeout (StandardBindingElement) results in quicker thread pool exhaustion
    • The thread/stack trace shows a large number of threads waiting in Microsoft.Adapters.OracleDB.OracleDBInboundContract.AdapterTryReceive

    Tools I have used during debugging:

    • SysInternals ProcessExplorer (thread count)
    • Microsoft Visual Studio (.net stack trace)
    • Reflector

    Software:

    • Windows Server 2003 R2 SP2 (32 bit)
    • BizTalk Server 2006 R2
    • WCF LOB Adapter SDK SP1
    • BizTalk Adapter Pack 1.0
    • ODAC 10g Release 2 (against Oracle 9i database server)

     

    Tuesday, October 21, 2008 1:21 PM

Answers

  • Hi everybody.

     

    I have found another workaround, that makes sure the problem doesn't grow over time. First of all, the receiveTimeout property is set to its maximum as described in the previous post. To make the receive location "recycle" every night, I've used the Service Window functionality, where the receive location is set to disable it self 2 minutes every midnight. Thus the threads are released once a day, and we doesn't get the accumulating threads over time.

     

    This is still a workaround, but actually makes the problem go away.

     

    Thanks for everybodys help.

     

    Mads.

     

    Monday, November 10, 2008 3:23 PM

All replies

  • hi,

     

    Looking at the software list, I assume you are using WCF LOB ORACLE adapter. I am not sure this setting applies to WCF adapter or not but on Enterprise ORACLE Adapter there is setting called "RefreshAgent" (http://msdn.microsoft.com/en-us/library/aa561962.aspx), if you set that to TRUE, it will stop and restart the working thread. I think that might resolve this issue.

     

    Interestingly enough, I have the same problem with enterprise oracle adapter and in my case the adapter is not even allowing me to set the property to TRUE. when I change the setting to TRUE and save the changes it reverts back to FALSE Sad I have posted the question on this forum with title "Enterprise ORACLE Adapter issues" and still trying to fix this issue. I will share the information if I find something.

     

     

    Tuesday, October 21, 2008 3:56 PM
  • Thanks for the response.

     

    Unfortunately, the setting "RefreshAgent" doesn't exist for the WCF LOB Oracle adapter, I am using.

     

    In the mean time I have found a workaround (which I'm sorry to say, will probably only work for the WCF LOB Oracle adapter). Setting the receiveTimeout property to its maximum value makes the issue tolerable. It seems for the WCF adapter the leaking of threads only happens when the receive operation times out, and thus the maximum setting for receiveTimeout of 24 days practically makes the issue go away.

     

    Still, I would prefer a fix to a workaround. BizTalk is still likely to run out of threads in the thread pool eventually - it just happens a whole lot slower than with the standard timeout of 10 minutes. With the standard setting BizTalk comes to a complete halt in a matter of a few days. Of course the issue multiplies with the number of Oracle polling receive locations.

     

    Mads.

    Wednesday, October 22, 2008 7:38 AM
  • Hi everybody.

     

    I have found another workaround, that makes sure the problem doesn't grow over time. First of all, the receiveTimeout property is set to its maximum as described in the previous post. To make the receive location "recycle" every night, I've used the Service Window functionality, where the receive location is set to disable it self 2 minutes every midnight. Thus the threads are released once a day, and we doesn't get the accumulating threads over time.

     

    This is still a workaround, but actually makes the problem go away.

     

    Thanks for everybodys help.

     

    Mads.

     

    Monday, November 10, 2008 3:23 PM