none
BizTalk FTP adapter is causing duplicate files RRS feed

  • Question

  • Hi,

    We are using BizTalk FTP receive location and set Delete After Download option to Yes and it is working fine mostly but once in a while it is failing to delete the file after download and causing duplicate files. Please throw some light.

    Error:

    The adapter "FTP" raised an error message. Details "Unable to delete the file "XXXX" on the FTP server. Inner Exception details: "The remote file could not be opened by the FTP server. ". ".

    Wednesday, May 11, 2016 2:26 PM

Answers

  • Basically, this is not a problem with BizTalk Server, or your app.

    This happens because FTP is not a transnational protocol and there are often transient circumstances on the wire or on the server that cause the file to not be deleted.  The most common is case is something on the FTP Server's side holding a lock on the file so the delete command is blocked or ignored.  Many FTP Servers do not properly communicate this to the client.

    Unfortunately, there isn't much you can do other than A) build logic in you app to deal with possible dups or B) don't use FTP.

    If this is a regular problem, you might consider a PowerShell script that downloads and checks that the delete was successful before copying to a NTFS Folder where BizTalk is watching.

    • Proposed as answer by Angie Xu Monday, May 23, 2016 2:08 AM
    • Marked as answer by Angie Xu Monday, May 23, 2016 2:08 AM
    Wednesday, May 11, 2016 2:44 PM
    Moderator
  • Hello,

    This could happen if the original document is still being written to the FTP server by the host application, the FTP adapter cannot delete the document and will retrieve another copy of the document at the next polling interval that is configured for the receive location. This behavior causes document duplication to occur. 

    Workaround could be:

    • Configure the host application to write to a temporary folder on the same hard disk as the public FTP folder and to periodically move the contents of the temporary folder to the FTP folder. The temporary folder should be on the same hard disk as the public FTP folder to make sure that the move operation is atomic. An atomic operation is an operation that is functionally indivisible. If you write data to the public FTP folder by using the BizTalk Server FTP adapter, you can do this by specifying a Temporary Folder property in the FTP Transport Properties dialog box when you configure a send port. If you specify a Temporary Folder property, make sure that this folder is on the same physical disk as the public FTP folder.

    • Configure the FTP receive location to operate within a service window when the host application is not writing data to the FTP server. You can specify the service window when you configure the receive location properties.

    Refer: Known Issues with the FTP Adapter

    You can also try the suggestion made here: FTP Adapter in BizTalk Server 2010:: The “Interval” and “Redownload Interval” Binding Properties


    Rachit Sikroria (Microsoft Azure MVP)


    Wednesday, May 11, 2016 5:42 PM
    Moderator

All replies

  • Basically, this is not a problem with BizTalk Server, or your app.

    This happens because FTP is not a transnational protocol and there are often transient circumstances on the wire or on the server that cause the file to not be deleted.  The most common is case is something on the FTP Server's side holding a lock on the file so the delete command is blocked or ignored.  Many FTP Servers do not properly communicate this to the client.

    Unfortunately, there isn't much you can do other than A) build logic in you app to deal with possible dups or B) don't use FTP.

    If this is a regular problem, you might consider a PowerShell script that downloads and checks that the delete was successful before copying to a NTFS Folder where BizTalk is watching.

    • Proposed as answer by Angie Xu Monday, May 23, 2016 2:08 AM
    • Marked as answer by Angie Xu Monday, May 23, 2016 2:08 AM
    Wednesday, May 11, 2016 2:44 PM
    Moderator
  • Hello,

    This could happen if the original document is still being written to the FTP server by the host application, the FTP adapter cannot delete the document and will retrieve another copy of the document at the next polling interval that is configured for the receive location. This behavior causes document duplication to occur. 

    Workaround could be:

    • Configure the host application to write to a temporary folder on the same hard disk as the public FTP folder and to periodically move the contents of the temporary folder to the FTP folder. The temporary folder should be on the same hard disk as the public FTP folder to make sure that the move operation is atomic. An atomic operation is an operation that is functionally indivisible. If you write data to the public FTP folder by using the BizTalk Server FTP adapter, you can do this by specifying a Temporary Folder property in the FTP Transport Properties dialog box when you configure a send port. If you specify a Temporary Folder property, make sure that this folder is on the same physical disk as the public FTP folder.

    • Configure the FTP receive location to operate within a service window when the host application is not writing data to the FTP server. You can specify the service window when you configure the receive location properties.

    Refer: Known Issues with the FTP Adapter

    You can also try the suggestion made here: FTP Adapter in BizTalk Server 2010:: The “Interval” and “Redownload Interval” Binding Properties


    Rachit Sikroria (Microsoft Azure MVP)


    Wednesday, May 11, 2016 5:42 PM
    Moderator