none
Performance issue - receiving IDOC's from SAP to BizTalk RRS feed

  • Question

  • Hi All,

    We've got a performance issue when SAP sends big number of IDOC's to BizTalk Server.
    Checking timestamp in Health and Activity Tracking shows that BizTalk has never received more than one IDOC a time through the same receive location. It receives in parallel using two receive locations though.
    Considering it takes approx. 700ms per RFC, it becomes a real bottleneck when sending lots of IDOC's.
    What are we missing to get IDOC's via RFC to the same destination (ProgramId) in parallel?

    SAP: smqs configures BizTalk destinations to use up to 50 max connections.
    Adapter binding: enableBizTalkCompatibilityMode=True; enableConnectionPooling=True; maxConnectionsPerSystem=50; receiveTimeout=23:59:59

    SAP ECC 6.0 Unicode (physical)
    SAP Client: SAP RFC SDK 7.0 UNICODE

    BizTalk environment (virtual):
    BizTalk Server 2006 R2
    BizTalk Adapter Pack (WCF based SAP Adapter v3.0)

    SQL Server (physical):
    MS SQL Server 2005

    Any ideas are very much appreciated.

    Regards,
    Nick

    Nick
    Friday, March 13, 2009 4:39 AM

Answers

  • 1. On a single connection to SAP, it is the SAP RFC SDK's design limitation that only one IDoc can be received at a time.
    2. There is a configuration setting on SAP, somewhere, (I don't know where) which controls whether if there are multiple connections to SAP using the same ProgramID, then will IDocs be sent out parallely over all the connections, or will they yet be serialized over the connections in a round robin fashion (e.g., IDoc1 goes to Connection1, and only when that completes, does IDoc2 go to Connection1, and when that completes, IDoc3 goes to Connection3, etc).

    Since one Receive Location corresponds to just one connection with SAP - if you want multiple connections, you'll have to create multiple Receive Locations. You can even create multiple Receive Locations for a single Receive Port for the same host/host instance - just change your URIs from:
    sap://blah/blah?ListenerProgramId=xyz
    to
    sap://blah/blah?ListenerProgramId=xyz&Id=1
    sap://blah/blah?ListenerProgramId=xyz&Id=2
    sap://blah/blah?ListenerProgramId=xyz&Id=3
    etc so that the URIs are unique for each Receive Location.
    • Marked as answer by nbusy Monday, March 16, 2009 11:09 AM
    Monday, March 16, 2009 12:36 AM

All replies

  • Hi!
    You can configure the ProgramId you're using on the SAP system to send multiple IDocs at one go (currently that is set to 1 I think) (Your SAP administrator will help you with that). Then you'll start getting more than 1 IDocs in one message.
    Thanks,
    Manas
    Friday, March 13, 2009 6:35 AM
  • Receiving one incoming call (which might contain a single IDoc, or multiple within the same call) at a time on a Receive Location is due to the design of the SAP RFC SDK Library and/or SAP.

    What you can try doing - create a Receive Location with some Program Id, and then create two BizTalk host instances for the host, and start both the host instances. You should now have the Receive Location running twice - once in each process corresponding to one host instance. Now send multiple IDocs from SAP - do you see both these Receive Locations (which are listening on the same ProgramId) receiving the IDocs in parallel?
    Saturday, March 14, 2009 12:29 AM
  • Manas & Mustansir,
    Thanks for the reply.

    Manas,
    As I said, we configured to use multiple simultaneous connections for the destination in SAP transaction smqs. Do you know another transaction number for more settings about multi-threaded run, or are you talking about sending multiple IDOC's (same type) in one message?
    The latter (pack size) requires "Collect IDocs" output mode with a background task configured. We use "Transfer IDoc immediately" mode as during business hours we need most of the IDocs in real time.

    Mustansir,
    BizTalk Server does not allow to run multiple host instances for the same host on the same server and we currently use only one production license. Even if I configured BizTalk that way, it wouldn't make much difference (two msgs vs one). To utilize servers resources efficiently we'd prefer multi-threaded mode with ability to configure max number of threads. I'm talking about at least 10-20 simultaneous threads.

    The first question would be - is it the BizTalk SAP Adapter limitation or SAP itself limitation or anything else? Or may be there is no that sort of limitation at all (because I couldn't find any reference on it in documentation) and we are just missing a thing.

    Regards,

    Nick
    Sunday, March 15, 2009 10:59 PM
  • 1. On a single connection to SAP, it is the SAP RFC SDK's design limitation that only one IDoc can be received at a time.
    2. There is a configuration setting on SAP, somewhere, (I don't know where) which controls whether if there are multiple connections to SAP using the same ProgramID, then will IDocs be sent out parallely over all the connections, or will they yet be serialized over the connections in a round robin fashion (e.g., IDoc1 goes to Connection1, and only when that completes, does IDoc2 go to Connection1, and when that completes, IDoc3 goes to Connection3, etc).

    Since one Receive Location corresponds to just one connection with SAP - if you want multiple connections, you'll have to create multiple Receive Locations. You can even create multiple Receive Locations for a single Receive Port for the same host/host instance - just change your URIs from:
    sap://blah/blah?ListenerProgramId=xyz
    to
    sap://blah/blah?ListenerProgramId=xyz&Id=1
    sap://blah/blah?ListenerProgramId=xyz&Id=2
    sap://blah/blah?ListenerProgramId=xyz&Id=3
    etc so that the URIs are unique for each Receive Location.
    • Marked as answer by nbusy Monday, March 16, 2009 11:09 AM
    Monday, March 16, 2009 12:36 AM
  • Mustansir,

    Thanks a lot for your help. Multiple receive locations with a fake parameter have done the trick.

    I tried to create 5 locations first and it improved performance at 5 times (as supposed). Then I tried to add 5 more but performance didn't change dramatically.
    We used two different programId for normal and high volume interfaces, so I completed the same sort of scenario for the second programId.
    Finally got 10 receive locations * 2 programId = 20 locations connected to SAP.
    Here it became very unstable and unpredictable. Some connections terminated before completing posting, which resulted "Message body was not read" error for a response message back to SAP.

    I reckon, it requires some fine tuning to determine the number of locations it can handle stably. Any consideration about it, or perhaps registry entry to maintain it?

    "1. On a single connection to SAP, it is the SAP RFC SDK's design limitation that only one IDoc can be received at a time."
    Can you point me to any documentation stating that? Whatever I could find did not include the limitation.

    Thank you again, your help is very much appreciated.
    Regards,

    Nick
    Monday, March 16, 2009 11:09 AM
  • On a single connection to SAP, it is the SAP RFC SDK's design limitation that only one IDoc can be received at a time.
    This is not mentioned as a limitation, since in reality, it is the design. If you refer to the SAP RFC SDK documentation and look at the APIs for implementing a RFC Server in C, you'll realize that this is how the API are designed. There is a strict request/response API pair which need to be called in sequence; and so until one request/response hasn't completed, you cannot accept the second request from SAP.
    Monday, March 16, 2009 11:23 AM
  • 1. On a single connection to SAP, it is the SAP RFC SDK's design limitation that only one IDoc can be received at a time.
    2. There is a configuration setting on SAP, somewhere, (I don't know where) which controls whether if there are multiple connections to SAP using the same ProgramID, then will IDocs be sent out parallely over all the connections, or will they yet be serialized over the connections in a round robin fashion (e.g., IDoc1 goes to Connection1, and only when that completes, does IDoc2 go to Connection1, and when that completes, IDoc3 goes to Connection3, etc).

    Since one Receive Location corresponds to just one connection with SAP - if you want multiple connections, you'll have to create multiple Receive Locations. You can even create multiple Receive Locations for a single Receive Port for the same host/host instance - just change your URIs from:
    sap://blah/blah?ListenerProgramId=xyz
    to
    sap://blah/blah?ListenerProgramId=xyz&Id=1
    sap://blah/blah?ListenerProgramId=xyz&Id=2
    sap://blah/blah?ListenerProgramId=xyz&Id=3
    etc so that the URIs are unique for each Receive Location.
    The setting is  'Pack. Size' in Outbounding Options (SAP).
    MOSS2007,BizTalk2006
    Friday, November 11, 2011 1:08 PM