File pick up speed decreasing in BizTalk host instances

Question

Hi,

I am using SFTP adapter to pull files from sftp location, I am using File delete on success mode.

In my source there can be 1000+ files size between 2-10 MB.

Max file count : 1

Polling interval : 5 sec

we did these polling settings to avoid deadlock while inserting into DB.

issue: first 30 min files processing as expected, after that file pick up speed slow down to 2 min for each file.

After restarting host instances observed normal process, but again after sometime same issue.

I have created separate host instances for this application, but still i'm having this issue

Please advise

Thanks

Answer 1

Hi 

Are you getting any warning messages in the BizTalk Server event log? You might be expericening throttling at this stage. 

Please have a look of this blog post

BizTalk 2013 SFTP Adapter – “BizTalk host Receiving throttled because ProcessMemory exceeded the configured throttling limit.”

---

Cheers,HTH

Naushad Alam http://alamnaushad.wordpress.com/

When you see answers and helpful posts, please click Vote As Helpful , Propose As Answer , and/or Mark As Answer

Answer 2

Hi,

Two things:

1) Polling SFTP location in an interval of every 5 sec will result in throttling and memory leak conditions.  You should be careful to limit its polling frequency by configuring the polling interval to be in minutes rather than seconds and to only run during scheduled times. Refer: https://connectedpawns.wordpress.com/2014/04/13/biztalk-2013-sftp-adapter-biztalk-host-receiving-throttled-because-processmemory-exceeded-the-configured-throttling-limit/

2) If you are on BizTalk Server 2013 R2, you should apply this hotfix on the servers, it is part of CU1: https://support.microsoft.com/en-us/help/3061233/fix-high-cpu-usage-when-you-use-the-sftp-adapter-s-default-polling-interval-in-biztalk-server-2013-r2

---

Rachit Sikroria (Microsoft Azure MVP)

Answer 3

we did these polling settings to avoid deadlock while inserting into DB.

SFTP Polling and DB Deadlocks have nothing to do with each other?

So, it seems you're solving the wrong problem by slowing down the SFTP Receive.

You're real problem is the database insert processes.  That's what you need to fix.

Answer 4

You are most certainly running into some type of throttling. 

When you are facing this issue, run Perfmon Counters to check the type of throttling you are running into.

Based on that, you can take remediation steps to mitigate this.

Also, make sure that the SFTP receive location runs in its own dedicated Host.

---

Thanks Arindam

Answer 5

There have been some good answers in the forum but i would also add that the Max settings does not work as you may think. The adapter will still get all the files that exists on the sftp at the moment :-(.

So up the settings for polling interval. there should be no deadlocks.

Make sure that you do as little processing in the pipeline as possible.

Answer 6

Hi,

you can easily get detailed insight into the situation with using the free version of our monitoring platform AIMS (www.aimsinnovation.com/aims-free). AIMS extracts performance data for all BizTalk hosts and every port & orchestration, and you can then easily chart and correlate the data to isolate a specific issue. You could try to chart performance data for the port (message count, latency etc) using the SFTP adapter plus the corresponding host and see what you find.

Thanks,

Marius