locked
Host Throttling on receiving files using SFTP RRS feed

  • Question

  • Hello,

    We have this problem in BTS2006 and now we migrated to BTS2013 and we still have the problem.

    Problem: We are using nSoftware SFTP adapter. Sometimes the host handler gets into idle state on some receive locations. It will not pickup the file until I restart the host instance. We have several RLs running on that particular host instance and they all are getting bounced when we restart the host instance. This is happening only on SFTPs. There is no guarantee that the file is delivered successfully. Most of the times business calls us stating the file is not received. The host create 64/32 bit if that matters. We have almost 200 interfaces running on each host instances not sure if that matters. I can create more hosts if needed but that would cost some system resources  and thinking of not to do.

    Any help will be greatly appreciated.

    Thanks in advance.

    Tuesday, October 21, 2014 11:58 PM

Answers

  • Since it might be inherited with the software/or volume you migrated from 2006 to 2013, it won't solve automatically just migrating BizTalk version. 
    Did you find out the throttling condition? I would suggest to have a look in to the link http://msdn.microsoft.com/en-au/library/aa559893.aspx which could be the starting point to look into for detecting the catch. 
    I have personally not used nSoftware SFTP adapter but I would suggest using separate host handler for suspected (victim) SFTP send ports and check the figures as per the conditions you identified. Since you already have 2013, it has very nice dashboard (right click on the host and go to setting) for you to play with figure. 
    If you share your volume/size of message and what the core process is, probably you would get better suggestion.
    Also worth looking into Host Throttling Performance Counters.


    Please mark it as Answer if this answers your question
    Thanks.
    Mo
    The contents I write here is my personal views, not the view of my employer and anyone else.

    • Proposed as answer by Angie Xu Monday, October 27, 2014 1:45 AM
    • Marked as answer by Angie Xu Wednesday, November 5, 2014 1:35 AM
    Wednesday, October 22, 2014 2:19 AM

All replies

  • Since it might be inherited with the software/or volume you migrated from 2006 to 2013, it won't solve automatically just migrating BizTalk version. 
    Did you find out the throttling condition? I would suggest to have a look in to the link http://msdn.microsoft.com/en-au/library/aa559893.aspx which could be the starting point to look into for detecting the catch. 
    I have personally not used nSoftware SFTP adapter but I would suggest using separate host handler for suspected (victim) SFTP send ports and check the figures as per the conditions you identified. Since you already have 2013, it has very nice dashboard (right click on the host and go to setting) for you to play with figure. 
    If you share your volume/size of message and what the core process is, probably you would get better suggestion.
    Also worth looking into Host Throttling Performance Counters.


    Please mark it as Answer if this answers your question
    Thanks.
    Mo
    The contents I write here is my personal views, not the view of my employer and anyone else.

    • Proposed as answer by Angie Xu Monday, October 27, 2014 1:45 AM
    • Marked as answer by Angie Xu Wednesday, November 5, 2014 1:35 AM
    Wednesday, October 22, 2014 2:19 AM
  •  We have almost 200 interfaces running on each host instances not sure if that matters.

    I don't think the issue stands with your nSoftware SFTP Adapter. The issue you are facing is more to do with this statement above.

    Hosts should be separated based on functionality for receiving, processing, sending and tracking.

    By not grouping areas of functionality, thread starvation can occur. In an example for receive and send processing occurring within the same host instance, if the host instance is using threads for receiving , threads will starve within the host instance for transmitting messages and become a bottleneck.

    Multiple BizTalk hosts provide flexibility when configuring the workload in a BizTalk group. This reduces contention on the MessageBox database host queue tables as each host is assigned its own work queue tables in the MessageBox database.

    I would strongly advise you to distribute your Biztalk artifacts based on receiving, processing, sending and tracking functionality and you will instantly observe the performance boost.

    Rachit

    If this answers your question please mark it accordingly. If this post is helpful, please vote as helpful by clicking the upward arrow mark next to my reply.


    Wednesday, October 22, 2014 4:02 AM
    Moderator
  • Yes you need to create separate host to eliminate the Host starvation scenarios where threads wont be available to process your request . I purposely would not support to have 200 Application running on one host  .

    Once you create a new host ,it creates you a working Queue inside BizTalk Message Box Db so that your Load can be balanced . There are threads configuration on host as  well which has a max limit depending on the cluster Environment .

    So better to Create new host instance for SFTP and analyze the performance after that .

    Thanks

    Abhishek
    Wednesday, October 22, 2014 4:40 AM
  • I can create more hosts if needed but that would cost some system resources  and thinking of not to do.

    The 'resources' used by an additional Host Instance are one of the last things you should even think about, meaning, don't worry about it, don't even let it cross your mind.

    Wednesday, October 22, 2014 11:22 AM
    Moderator
  • Thanks Guys...  

    I have created 6 host new hosts and create 3 instances on one server another 3 on 2nd app server. Allocated them to only receive handlers. I'm trying to limit the number of receive locations to below 100 on each host. As for throttling numbers, I kept all of the hosts on defaults not sure if I have to increase the CPU thread count. It currently has zero which normal.

    Right now, we have 2 app servers  and 18 host and one isolated host... all of them have a default settings.

    This is really creating a nightmare as I have no idea if worked or not until business notifies me. Frustrating... I think I need to understand the throttling little bit better:)

    Thanks guys again... let's see if this will work or not. I will update you after I change the receive handler on RLs this weekend.


    Wednesday, October 22, 2014 5:20 PM