locked
Messages being sent multiple times by a file send port RRS feed

  • Question

  • Hello All,

    I am using a file send port to send a message to a NAS location. Every now and then I get an issue where the file is being sent twice to the NAS location. When I checked message flow in BizTalk I saw that the message is being sent twice even when it was sent successfully the first time. The screenshot of the same is attached too.

    Can anyone please explain why is this happening?

    Friday, November 22, 2013 1:26 PM

Answers

  • When dealing with writes across WAN/LAN, sometimes the return transport Ack does not make it through so even though the file did get through, BizTalk File Adapter did not get the Ack and thus goes in for Retry.

    On the send port in question can you disable retries? This way if the file did get through and the ACK did not, BizTalk would suspend the message. You can then reconcile the behaviour.

    The other thing when dealing with network locations is to use UNC paths in the File Adapter as opposed to the local mapped drive paths. E.g.: Assume a NAS Location \\NASSERVER\NASLOCATION... you could have the \\NASSERVER mapped as Z: on the server and in the Port use Z:\NASLOCATION as the destination. This is not correct and could cause resource contention (security token) in case the same location is referred by another send port running under a different host instance. The correct way would be to use the complete path \\NASSERVER\NASLOCATION\*.txt in the file adapter configuration.

    Regards.

    Friday, November 22, 2013 1:57 PM
  • The transmission failure is when BizTalk File Adapter did not get the Ack for the message transmission while the OK is for the second attempt when the ack actually made it through.

    I was not talking about file extensions, I was talking about folder paths. connected drives have a tendency to timeout due to inactivity while UNC Paths do not suffer from this issue as the connection is established when the send is requested.

    Regards.

    Friday, November 22, 2013 2:17 PM
  • You read the Message Flow from the perspective of the Pipeline Service.

    The first step is the delivery of the Message to the Pipeline Service.  Note the little envelope with the arrow pointing > in.  That OK means the message was delivered to the Pipeline, which in practice should never really fail.

    The second and third steps are the essentially the Adapter's attempt to deliver the Message, having already been processed by the Pipeline.  Here, see the little envelope with the arror pointing out >.

    The first attempt failed, the second attempt succeeded.

    How big is the message and how fast is that network/NAS?  The stream was likely transmitted twice.

    Friday, November 22, 2013 2:45 PM
    Moderator
  • On the FILE Adpater dialog, General tab.

    "Use temporary file while writing"

    Friday, November 22, 2013 3:21 PM
    Moderator

All replies

  • When dealing with writes across WAN/LAN, sometimes the return transport Ack does not make it through so even though the file did get through, BizTalk File Adapter did not get the Ack and thus goes in for Retry.

    On the send port in question can you disable retries? This way if the file did get through and the ACK did not, BizTalk would suspend the message. You can then reconcile the behaviour.

    The other thing when dealing with network locations is to use UNC paths in the File Adapter as opposed to the local mapped drive paths. E.g.: Assume a NAS Location \\NASSERVER\NASLOCATION... you could have the \\NASSERVER mapped as Z: on the server and in the Port use Z:\NASLOCATION as the destination. This is not correct and could cause resource contention (security token) in case the same location is referred by another send port running under a different host instance. The correct way would be to use the complete path \\NASSERVER\NASLOCATION\*.txt in the file adapter configuration.

    Regards.

    Friday, November 22, 2013 1:57 PM
  • Hello Shankycheil,

    First of all thanks a lot for the prompt reply.

    Usually this flow is after the office hours, so if I disable the retries, it might cause an issue. So, would it help if I add the extension of the file, say .xml in the file masking i.e instead of using %sourcefilename%, I use %sourcefilename%.xml or something?

    Friday, November 22, 2013 2:08 PM
  • Also, as in the status we can see that the value is OK, so doesn't that mean that the Ack has been received by BizTalk?

    In the screenshot there are 2 status: OK and Transmission Failure, so can you please explain when the cases when we would get these 2 status?

    Friday, November 22, 2013 2:10 PM
  • The transmission failure is when BizTalk File Adapter did not get the Ack for the message transmission while the OK is for the second attempt when the ack actually made it through.

    I was not talking about file extensions, I was talking about folder paths. connected drives have a tendency to timeout due to inactivity while UNC Paths do not suffer from this issue as the connection is established when the send is requested.

    Regards.

    Friday, November 22, 2013 2:17 PM
  • But if we see the first time also we have received a OK status in the message then why was the message sent again? (There are a total of 3 tries: 1 ok, 2: transmission failure, 3: OK)

    Also can you please explain why the end time in the details set so late (23:38) even when the message has received an OK status at around 23:12

    Friday, November 22, 2013 2:26 PM
  • You read the Message Flow from the perspective of the Pipeline Service.

    The first step is the delivery of the Message to the Pipeline Service.  Note the little envelope with the arrow pointing > in.  That OK means the message was delivered to the Pipeline, which in practice should never really fail.

    The second and third steps are the essentially the Adapter's attempt to deliver the Message, having already been processed by the Pipeline.  Here, see the little envelope with the arror pointing out >.

    The first attempt failed, the second attempt succeeded.

    How big is the message and how fast is that network/NAS?  The stream was likely transmitted twice.

    Friday, November 22, 2013 2:45 PM
    Moderator
  • The file was around 200kb. The issue that happened was that BizTalk had sent the file twice to the NAS location. First time half of the file was sent i.e it wasn't sent completely while second time the file was sent completely with all the tags.
    Friday, November 22, 2013 2:56 PM
  • Is there a way where BizTalk sends the file as a temp file to the NAS location and only after it has been completely written BizTalk changes the extension to .xml?

    Friday, November 22, 2013 3:11 PM
  • On the FILE Adpater dialog, General tab.

    "Use temporary file while writing"

    Friday, November 22, 2013 3:21 PM
    Moderator
  • Currently in the send port configuration I have set the copy mode as Override. But I don't see a point to it as the retry interval that I have now is for 5 minutes and till that time the file is picked up by the destination application for further use.

    So, in order to apply the temp logic I will have to change the property of copy mode to create new and then select the use temporary file while writing. This way the destination system will only pick up the .xml files and hence wont pick up incomplete files. But this might lead to another issue. like when we try to send a file and it does not reach the destination system completely it will remain in .inprogress extension and after 5 minutes the file will be pushed again as a new file. so this way there will be an incomplete .inprogress file left at the destination location.

    Is there anyway to handle that?

    Friday, November 22, 2013 3:26 PM
  • Yes, the scenario you describe is plausible.  But to be perfectly honest, this isn't your or BizTalk's problem.

    You need to raise the issue to the network/NAS team to find out why their end isn't reliably receiving files.

    Friday, November 22, 2013 3:38 PM
    Moderator
  • What is the time taken to transfer file to the NAS. Measure that because that would give you an idea of what to set your retry interval as.. if the file takes more than 5 minutes to get transferred and your retry interval is 5 minutes, if the ACK does not get through, BizTalk will consider it as a failure and retry.

    As boatseller mentions, get in touch with the network team tofigure out just why it takes so long to get the file across reliably.

    Regards.

    Saturday, November 23, 2013 5:52 AM