none
Biztalk 2013 R2 SFTP download duplicates RRS feed

  • Question

  • We noticed the Biztalk 2013 R2 SFTP adapter downloading duplicates from the SFTP location. when this happened, there was a network network hiccup on the server.  

    Any recommendations to prevent duplicates from SFTP adapter would be greatly appreciated. 


    Venkat Kundavaram

    Monday, July 17, 2017 11:12 PM

All replies

  • The SFTP adapter in BizTalk still lacks some as the temp folder for transactional processing is missing on receiving

    One option is to use a third party adapter such as the free SFTP adapter from blogical
    https://github.com/wmmihaa/BizTalk-Sftp-Adapter/tree/master/Blogical.Shared.Adapters.Sftp.Schemas

    hth /Peter

    Tuesday, July 18, 2017 8:02 AM
  • Venkat,

    There is no issue on the adapter process here. Adapter retrieves file to the BizTalk Messaging Engine for processing and then it deletes the original file from the FTP server. This happens using communications (between SFTP client-BizTalk Adapter and SFTP server) with command like "SSH_FXP_OPENDIR" and "SSH_FXP_HANDLE" Until this process to complete adapter expect network connection to the server active. May be in your case, the network issues cases adapter not to finish its process completely i.e retrieval and delete from SFTP server and the SFTP client (BizTalk adapter). When the network issue resolves new thread from the adapter tries to retrieve the file again from the server.

    SFT adapter doesn't have feature like “Delete After Download” and “Enable Timestamp comparison” as its available in FTP adapter where problem like this case could be avoidable.

    Best bet in your case is solve the root cause which is the network issue rather than addressing the symptom which is the duplicate issue



    If this answers your question please mark it accordingly. If this post is helpful, please vote as helpful by clicking the upward arrow mark next to my reply.

    Tuesday, July 18, 2017 9:59 AM
  • Unfortunately network issues are not always resolvable 

    /Peter

    Tuesday, July 18, 2017 10:10 AM
  • For clarity, this is not a problem with you, your app or BizTalk Server.

    This is an existing, since forever, well known gap in the various FTP protocols.  Basically, there is no concept of a 'transaction' though some servers handle this better than others.

    The last time I had such issues, it was due to a very quirky server implementation. Instead of using the SFTP Adapter directly, we created a script using WinSCP that handled the server issues much better. The script was run periodically by the SQL Agent.

    Tuesday, July 18, 2017 1:17 PM
    Moderator
  • Are you running the SFTP adapter on more than one host instance at the same time?

    From Considerations for Running Adapter Handlers within a Clustered Host:

    "For most of the BizTalk Server integrated adapters, high availability can be achieved by creating multiple adapter handlers to run on BizTalk host instances on different BizTalk Server servers within a BizTalk group. FTP or SFTP adapter receive handlers should not, however, be configured to run in multiple BizTalk host instances simultaneously. This recommendation is made because the FTP or SFTP receive adapter uses the FTP or SFTP protocol to retrieve files from the target system. The FTP or SFTP protocol does not lock files to ensure that multiple copies of the same file are not retrieved simultaneously when running multiple instances of the FTP or SFTP receive adapter. To provide high availability for the FTP or SFTP receive adapter, you should configure the FTP or SFTP receive adapter to run in a BizTalk host instance that has been clustered."


    • Edited by twhensley Friday, July 21, 2017 5:05 PM
    Friday, July 21, 2017 5:01 PM
  • I am using single host instance for SFTP but still duplicate files are being downloaded from SFTP in BizTalk 2013 R2.

    Any recommendations to prevent duplicates from SFTP adapter ?

    Tuesday, July 2, 2019 11:00 AM
  • Hi Juhi_Gupta

    As others have said you may need to use a third party adapter that supports the temporary filename or folder feature.

    The only other option is to upgrade to BizTalk 2016 where they have fixed many SFTP issues, including FIX: Duplicate files received by SFTP adapter when long time is taken for message processing in BizTalk Server in CU6

    Wednesday, July 3, 2019 11:04 PM