none
Outlook 2003's use of multiple TCP connections RRS feed

  • Question

  • I observed the following behavior with Outlook 2003. The server was Exchange 2003.

    - Set up TCP connection C1
    - Set up a sync stream over C1, with handle H
    - Read the stream over C1 using RopFastTransferSourceGetBuffer's and handle H
    - In the middle of reading the stream, create a new TCP connection, C2
    - Read the stream over C2 using RopFastTransferSourceGetBuffer's and handle H
    - Read the stream back over C1 using RopFastTransferSourceGetBuffer's
      and handle H

    I did not expect this behavior at all. Looks like MSRPC lets clients
    multiplex operations over multiple TCP connections, and handles opened
    over one TCP connection are valid on the others.

    - Is this explained somewhere in your Exchange/MSRPC documents? Looks
      like I need to go back and understand the association between TCP
      streams that outlook creates.
    - Do Outlook 2007 and 2010 also exhibit the same behavior?
    - Is the server free to send the response for the request on one TCP
      connection over a different (and associated) TCP connection?

    Thanks,

    Hari

    Monday, January 24, 2011 7:15 PM

Answers

  • Hari,

     

    DCE 1.1 RPC C706 specification provides the answers to these two questions. Note that I am using C706 terminology of association and association group.

     

    Question:

     

    To simplify the question, when the client closes all the connections in an association group, does the server delete that association group?

     

    Answer:

     

    Association management policy is implementation-specific with respects to the constraints defined in [C706]. Association group and association management are defined in [C706] (see sections 9.3.2, 9.3.3 for details).

    As specified in [C706] Section 9.4.4, an instance of CO_SERVER_GROUP protocol machine corresponding to an association group is terminated when the last association in the group terminates and none of the remaining context handles is needed. When an instance of CO_SERVER_GROUP terminates, the group ID associated with this group is no longer valid, as specified in [C706] Section 11.5.1 CO_SERVER_GROUP States.

     

    Question:

     

    A related question is: If the server receives a assoc-group-id in a bind request that does not exist on the server, does it fail the bind with bind_nak or return a new assoc-group-id in the (successful) bind response?

     

    Answer:

     

    It is a client protocol error to send an RPC bind request with an assoc_group_id which does not match the group ID of any association group on the server, as specified in DCE1.1 [C706] Section 11.5.1 CO_SERVER_GROUP States.

    Although [C706] describes this client behavior as a protocol error, it does not specify which reject reason to return. Therefore, the reject reason or returned error code is implementation specific.

    As an informational note, Windows servers fail such a bind with a bind_nak and a reject reason of REASON_NOT_SPECIFIED. If extended error is supported (e.g.  Windows server 2008 or 2008 R2) and is enabled, the bind_nak will have an extended error with status RPC_S_ENTRY_NOT_FOUND.

     

    References:

     

    DCE1.1 RPC [C706]

    9.3 Connection-oriented Protocol

    9.3.2 Association Group

    9.3.3 Association

    9.3.3.1 Association Management Policy

    9.4.4 CO_SERVER_GROUP

    11.5.1 CO_SERVER_GROUP States

     

    MS-RPCE

    2.2.2.5   New Reasons for Bind Rejection

    The following table briefly describes the reject reasons used by these extensions. These reasons are defined in [C706] section 12.6.3.1.

    Reject reason

    Value

    Meaning

    REASON_NOT_SPECIFIED

    0x00

    If the reason for the error does not fit any of the other reasons specified in this section, then this rejection code SHOULD be used.

     

    MS-ERREF

    2.2 Win32 Error Codes

     

    Win32 error codes

    Description

    0x000006E1
    RPC_S_ENTRY_NOT_FOUND

    The entry is not found.

     

    Regards,

    Edgar

    Tuesday, February 22, 2011 10:38 PM
    Moderator

All replies

  • Hi TH00:

    I have alerted protocol documentation team about your inquiry. A member of the team will be in touch soon through this thread.


    Regards, Obaid Farooqi
    Tuesday, January 25, 2011 6:43 PM
    Owner
  • Hari, I am the engineer who will be working with you on this issue. I am currently researching the problem and will provide you with an update soon. Thank you for your patience.
    Josh Curry (jcurry) | Escalation Engineer | US-CSS DSC Protocols Team
    Tuesday, January 25, 2011 7:31 PM
    Moderator
  • Thanks for working on this. Please add the following to your research
    list. 'flow' below is a TCP connection.

    Outlook 2003 - Exchange 2003
    - Session context handles (ie CXH) created on one flow are used in other flows.
      Server accepts them, so obvioulsy this works.
    - Some flows do not send EcDoConnectEx at all. They simply use CXH's
      created by other flows.
    - Some flows do not release the CXH they created.
    - An extension to the previous item - CXH created in one flow is closed
      in a different flow.

    - MS-OXCRPC does not seem to comment on the above.


    Outlook 2007 - Exchange 2007
    - As far as I can tell each flow seems to create its own CXH by sending
      EcDoConnectEx.
    - Outlook 2007 creates two flows in the beginning that seem to stay
      around for a long time. When you exit outlook, these two flows do not
      release their CXH. Looks like a bug.
      Question: Does the server clean up all the CXH's for a particular
      client when all the flows from that client are terminated?

    - I have turned off encryption. Outlook 2007 creates many short-lived MAPI flows. A couple of them are encrypted. Is this expected behavior?


    Question: Is the behavior of using CXH's across flows specific to
    Outlook 2003 or do Outlook 2007/2010 also do this?

    Thursday, January 27, 2011 6:20 PM
  • Hari, the key here is in how these TCP connections are established at a higher, RPC, level. Scoping works like this:

     

    ·         SOH (like handle H) are scoped to a logon.

    ·         Logons are scoped to RPC context handles.

    ·         RPC context handles are scoped to RPC association groups.

    ·         Physical connections (TCP for ncacn_http, HTTP pairs for ncacn_http) are also scoped to RPC association groups.

     

    This means that two TCP connections that were opened as a part of the same association group will be completely interchangeable. More information about this can be found in the MS-RPCE protocol document.

     

    I hope this helps.
    Josh Curry (jcurry) | Escalation Engineer | US-CSS DSC Protocols Team
    Tuesday, February 1, 2011 7:05 PM
    Moderator
  • Thanks for the quick response Josh. I have a few sanity check questions. The answer to all the following statements should be 'yes'. Can you please confirm?

    • Let's say the client has created the combinations logon-1/context-1 and logon-2/context-2. The server is free to return the same value for server handles (created via different requests that use these combinations) in the above two combinations - i.e. the client cannot assume that server handle values are unique across all logon/context combinations.
    • If the client uses a particular logon/context combination to obtain a server handle, then for all operations on this handle, it MUST send the same logon/context to the server.
    • The restriction on monotonically increasing call-id's only applies to individual TCP connections and not to association groups.

    Thanks,

    Hari

    Thursday, February 3, 2011 6:40 PM
  • Hari, the first 2 statements are true. More information can be found in MS-OXCRPC and MS-OXCROPS. The third statement is false. Call IDs are per association group. They should increase monotonically for the association group, regardless of the connection they were made on. More information about that can be found on this page of the DCE 1.1 specification in the 'Connection-oriented PDU Data Types' section.
    Josh Curry (jcurry) | Escalation Engineer | US-CSS DSC Protocols Team
    Tuesday, February 8, 2011 8:25 PM
    Moderator
  • Thanks Josh. Have a couple of follow up questions:

    - Are TCP connections in the same association group interchangeable to the server also?

    - If the answer to the above is 'yes', let's say there are two TCP connections C1 and C2 in an association group. The server receives a request on C1. Is it free to send the response for this request on C2?

    - The server returns a 'session index' in EcDoConnectEx response that 's scoped to a session context. This session index is returned in RopPending. Similarly RopNotify is sent when notifications are pending for a session context. Since session context's are valid across all TCP connections that belong to an association group, is the server free to send RopNotify and RopPending over any of those TCP connections?

    Thanks,

    Hari
    Thursday, February 10, 2011 5:49 PM
  • For the second question, Section 9.3.3 in the DCERPC spec says:

    For each RPC, an association is allocated, the RPC is made, and the association is deallocated
    when the RPC completes.

    This seems to imply that the server must send the response over the same association. The same section also says that there is a 1-1 correspondence between associations and transport connections, so by extension, the response should be sent over the same connection over which the request was received.

    Is this interpretation correct?

    Thanks,

    Hari

    Monday, February 14, 2011 6:15 PM
  • A different question:

    I have not seen any specific mention in the DCERPE or MS-RPCE specs about the lifetime of RPC association groups other than this:

    The set of associations between a client and a
    server is represented by an association group. Association groups are managed by client and
    server association group machines (CO_CLIENT_GROUP and CO_SERVER_GROUP). The
    creation and lifetime of these various protocol machines is a function of resource availability, the
    relationships described in this section, external events and local system policy.

    This talks about the lifetime of 'protocol machines' (which from the description looks like can be indefinite) - not sure if this extends to the association groups managed by the protocol machines.

    To simplify the question, when the client closes all the connections in an association group, does the server delete that association group?

    A related question is: If the server receives a assoc-group-id in a bind request that does not exist on the server, does it fail the bind with bind_nak or return a new assoc-group-id in the (successful) bind response?

    Thanks,
    Hari

    Monday, February 14, 2011 6:16 PM
  • Hari,

    Thanks for your new questions. We will review these and follow-up soon.

    Regards,

    Edgar

    Monday, February 14, 2011 10:16 PM
    Moderator
  • Hari, I believe that you are correct about section 9.3.3 of the C706 spec. After reviewing that entire section myself I came to the same conclusion. A response should always be sent back on the connection that was used to send the request.

     

    Regarding the following question, "is the server free to send RopNotify and RopPending over any of those TCP connections?" That is an application (Exchange Server, or another application that implements the Exchange Server Protocols) behavior type of issue and is beyond the scope of the protocol documentation.


    Josh Curry (jcurry) | Escalation Engineer | US-CSS DSC Protocols Team
    Tuesday, February 22, 2011 10:26 PM
    Moderator
  • Hari,

     

    DCE 1.1 RPC C706 specification provides the answers to these two questions. Note that I am using C706 terminology of association and association group.

     

    Question:

     

    To simplify the question, when the client closes all the connections in an association group, does the server delete that association group?

     

    Answer:

     

    Association management policy is implementation-specific with respects to the constraints defined in [C706]. Association group and association management are defined in [C706] (see sections 9.3.2, 9.3.3 for details).

    As specified in [C706] Section 9.4.4, an instance of CO_SERVER_GROUP protocol machine corresponding to an association group is terminated when the last association in the group terminates and none of the remaining context handles is needed. When an instance of CO_SERVER_GROUP terminates, the group ID associated with this group is no longer valid, as specified in [C706] Section 11.5.1 CO_SERVER_GROUP States.

     

    Question:

     

    A related question is: If the server receives a assoc-group-id in a bind request that does not exist on the server, does it fail the bind with bind_nak or return a new assoc-group-id in the (successful) bind response?

     

    Answer:

     

    It is a client protocol error to send an RPC bind request with an assoc_group_id which does not match the group ID of any association group on the server, as specified in DCE1.1 [C706] Section 11.5.1 CO_SERVER_GROUP States.

    Although [C706] describes this client behavior as a protocol error, it does not specify which reject reason to return. Therefore, the reject reason or returned error code is implementation specific.

    As an informational note, Windows servers fail such a bind with a bind_nak and a reject reason of REASON_NOT_SPECIFIED. If extended error is supported (e.g.  Windows server 2008 or 2008 R2) and is enabled, the bind_nak will have an extended error with status RPC_S_ENTRY_NOT_FOUND.

     

    References:

     

    DCE1.1 RPC [C706]

    9.3 Connection-oriented Protocol

    9.3.2 Association Group

    9.3.3 Association

    9.3.3.1 Association Management Policy

    9.4.4 CO_SERVER_GROUP

    11.5.1 CO_SERVER_GROUP States

     

    MS-RPCE

    2.2.2.5   New Reasons for Bind Rejection

    The following table briefly describes the reject reasons used by these extensions. These reasons are defined in [C706] section 12.6.3.1.

    Reject reason

    Value

    Meaning

    REASON_NOT_SPECIFIED

    0x00

    If the reason for the error does not fit any of the other reasons specified in this section, then this rejection code SHOULD be used.

     

    MS-ERREF

    2.2 Win32 Error Codes

     

    Win32 error codes

    Description

    0x000006E1
    RPC_S_ENTRY_NOT_FOUND

    The entry is not found.

     

    Regards,

    Edgar

    Tuesday, February 22, 2011 10:38 PM
    Moderator
  • Edgar & Josh,

    Thanks for your replies & effort.

    Hari

    Wednesday, February 23, 2011 6:47 PM