locked
BiztTalk Server 2016 SFTP receive host instances load balancing RRS feed

  • Question

  • Hi,

    I have more than 150 sftp server and 6 integrated biztalk server 2016 but there is issue to get all files from different 150 server with the help of one sftp host instance and few are working sftp receive location automatically stopped.

    Pl provide me urgent solution of following query-

    1. Load balancing for sftp receive port

    2. Way to prevent to stop few sftp receive location.

    Friday, October 27, 2017 6:01 AM

All replies

  • You have 6 servers. YOu can have the sftp receive host running on all server. 

    Have you checked why some recives are stopping. It coul be netwrok issues as your receive polls for a set number of times before stopping. 


    Pi_xel_xar

    Blog: My Blog

    BizTalkApplicationDeploymentTool: BizTalk Application Deployment Tool/

    Friday, October 27, 2017 6:13 AM
    Answerer
  • When the configured error count has exceeded the receive location will stop, it's default behaviour

    Do have any system to collect eventlog data from your servers?
    Any raised errors in the receive locations will be reported in the application log

    /Peter

    Friday, October 27, 2017 6:54 AM
  • I cannot run receive host on all 6 server otherwise i shall receive same file in six time. As default design for FTP and SFTP we can run only one receive host instance so there is problem to get all files by one receive host instance. Pl let me other solution in biztalk to get all file without any interrupt.

    Friday, October 27, 2017 7:03 AM
  • When the configured error count has exceeded the receive location will stop, it's default behaviour

    Do have any system to collect eventlog data from your servers?
    Any raised errors in the receive locations will be reported in the application log

    /Peter

    If any error out of more than 150 sftp server then is any option or setting to avoid to stop sftp receive location when again sftp server is ready to start working its getting file from sftp server. Nothing is showing in application log.

    Friday, October 27, 2017 7:07 AM
  • The issue is with FTP and not SFTP. SFTP handles file differently than FTP. FOr SFTP you should be able to create additional host.

    Pi_xel_xar

    Blog: My Blog

    BizTalkApplicationDeploymentTool: BizTalk Application Deployment Tool/

    Friday, October 27, 2017 7:08 AM
    Answerer
  • Nothing is showing in application log.

    Weird
    In Biztalk 2016 you can configure a log file on the SFTP receive location

    To reenable you can schedule a powershell job or let your monitoring tools reenable
    Eg
    https://www.codeproject.com/Tips/1062097/Monitor-BizTalk-ports-suspended-messages-and-send

    Friday, October 27, 2017 7:28 AM
  • The issue is with FTP and not SFTP. SFTP handles file differently than FTP. FOr SFTP you should be able to create additional host.

    Pi_xel_xar

    Blog: My Blog

    BizTalkApplicationDeploymentTool: BizTalk Application Deployment Tool/


    No I have test and same also documented by microsoft in biztalk server 2016 at sftp receive side there is no load balancing.
    Friday, October 27, 2017 8:46 AM
  • we can run only one receive host instance so there is problem to get all files by one receive host instance. 

    Are we discussing "Host Instance" as a BizTalk Service one Server/Node or "Hosts" as the 6 BizTalk Nodes in your setup?

    You can easily have several BizTalk Services (Hosts/Host Instances) and distribute the SFTP load across multiple BizTalk Services

    /Peter

    Friday, October 27, 2017 9:01 AM
  • we can run only one receive host instance so there is problem to get all files by one receive host instance. 

    Are we discussing "Host Instance" as a BizTalk Service one Server/Node or "Hosts" as the 6 BizTalk Nodes in your setup?

    You can easily have several BizTalk Services (Hosts/Host Instances) and distribute the SFTP load across multiple BizTalk Services

    /Peter


    Rather than FTP and SFTP it working perfect for both side send or receive but in case of sftp or ftp if i have created one host and it sftp receive handler and 6 host instances for its then it pick same file by 6 times and one instance cannot handle load of all our sftp server.
    Friday, October 27, 2017 9:09 AM
  • In Platform Settings (BizTalk Server Administrator) you have the possibility to create multiple Host/Host Instances
    For an adapter you can have one or more Hosts configured

    For a receive location you have the option to choose which host to use
    Aka you have multiple services on one server each service handling a subset of receive locations

    Eg (Configuration on each BizTalk Server/Node)
    Hosts

    • SFTP-Receive-A
    • SFTP-Receive-B


    Host Instances

    • SFTP-Receive-A
    • SFTP-Receive-B


    Receive location

    • SFTP Receive Location 1 uses host SFTP-Receive-A
    • SFTP Receive Location 2 uses host SFTP-Receive-A
    • SFTP Receive Location 3 uses host SFTP-Receive-B
    • SFTP Receive Location 4 uses host SFTP-Receive-B


    /Peter

    Friday, October 27, 2017 9:31 AM
  • Hi Jitendra,

    The SFTP host needs to be clusteured if you want to have the HA for the SFTP receive locations. Nowe in order to create a clustured Host, you need to have the Biztalk application servers clustured in a windows failover cluster.

    Easiest way for you to manage is to treat one server at a time to receive the messages for one sftp location. That way the file will be picked up only once. Now you can distribute the load by creating multiple Host and  then distribute their host instances on 6 servers. Bind SFTP receive locations to these hosts, that way no one server will ever have to bear the load from all the sftp receive locations.

    Another point is that whenever the error threshold for the SFTP receive location exceeds it just disables itself.

    What you can do to counter is 

    1) Create a PS script which monitors the receive location status and restarts it if it is disabled.


    Mandar Dharmadhikari

    Friday, October 27, 2017 10:11 AM
    Moderator
  • In Platform Settings (BizTalk Server Administrator) you have the possibility to create multiple Host/Host Instances
    For an adapter you can have one or more Hosts configured

    For a receive location you have the option to choose which host to use
    Aka you have multiple services on one server each service handling a subset of receive locations

    Eg (Configuration on each BizTalk Server/Node)
    Hosts

    • SFTP-Receive-A
    • SFTP-Receive-B


    Host Instances

    • SFTP-Receive-A
    • SFTP-Receive-B


    Receive location

    • SFTP Receive Location 1 uses host SFTP-Receive-A
    • SFTP Receive Location 2 uses host SFTP-Receive-A
    • SFTP Receive Location 3 uses host SFTP-Receive-B
    • SFTP Receive Location 4 uses host SFTP-Receive-B


    /Peter

    Hi Peter,

    Thanks for your suggestion but in my case it is not possible if i will distribute all sftp server from my six biztalk server then each server have approx 30 receive end to end location and if one server will be down for a certain period of time its too much hectic for me to change host instance for it so Please give me any other solution.

    Friday, October 27, 2017 10:48 AM
  • Hi Jitendra,

    The SFTP host needs to be clusteured if you want to have the HA for the SFTP receive locations. Nowe in order to create a clustured Host, you need to have the Biztalk application servers clustured in a windows failover cluster.

    Easiest way for you to manage is to treat one server at a time to receive the messages for one sftp location. That way the file will be picked up only once. Now you can distribute the load by creating multiple Host and  then distribute their host instances on 6 servers. Bind SFTP receive locations to these hosts, that way no one server will ever have to bear the load from all the sftp receive locations.

    Another point is that whenever the error threshold for the SFTP receive location exceeds it just disables itself.

    What you can do to counter is 

    1) Create a PS script which monitors the receive location status and restarts it if it is disabled.


    Mandar Dharmadhikari

    Hi Mandar,

    Clustered host work on active and passive base it not load balance our process and i am able to download same file from winscp tool but same by using biztak server its now getting for few sftp server.

    Friday, October 27, 2017 10:52 AM
  • Read Mandar's post below

    With clustered servers the failover will happen instantly for all clustered resources in case of a server failure

    /Peter

    Friday, October 27, 2017 10:53 AM
  • Clustered hosts can be configured as active/active
    But yes it's a bit different than NLB

    Considerations for Running Adapter Handlers within a Clustered Host
    https://docs.microsoft.com/en-us/biztalk/core/considerations-for-running-adapter-handlers-within-a-clustered-host1

    /Peter

    Friday, October 27, 2017 10:58 AM
  • Ya correct. I was confused with the NSoftware SFTP. That works great across servers.

    Create multiple hosts and assign say 50 locations per host. (Your host will still run on 1 server but on different servers)


    Pi_xel_xar

    Blog: My Blog

    BizTalkApplicationDeploymentTool: BizTalk Application Deployment Tool/

    Friday, October 27, 2017 11:33 AM
    Answerer
  • Ya correct. I was confused with the NSoftware SFTP. That works great across servers.

    Create multiple hosts and assign say 50 locations per host. (Your host will still run on 1 server but on different servers)


    Pi_xel_xar

    Blog: My Blog

    BizTalkApplicationDeploymentTool: BizTalk Application Deployment Tool/


    I know but in this case it is not workable for when any one of them biztalk server will be shut down.
    Friday, October 27, 2017 12:23 PM
  • No It should if you make your host clustered. You have 6 servers , you can use 3 for receiving and 3 for the clustered one so that when one of those fails the othe rone takes over. 

    It would be similar to how you make your FTP host HA.


    Pi_xel_xar

    Blog: My Blog

    BizTalkApplicationDeploymentTool: BizTalk Application Deployment Tool/

    Friday, October 27, 2017 12:28 PM
    Answerer