none
[MS-TDS] Understanding RPC Request RRS feed

  • Question

  • Hi,

    I've read the specification of RPC Request (multiple times), but I still fail to "parse" correctly the traffic that I see. So I'd like to give an example and to ask number of related questions.

    The protocol in this example is TDS 7.1 (SQL Server 2000).

    00:  03 01 00 d3 00 00 01 00   ff ff 0d 00 00 00 00 01
    10:  26 04 04 ff ff ff ff 00   00 63 00 00 00 00 09 04
    20:  d0 00 34 ff ff ff ff 00   00 63 9c 00 00 00 09 04
    30:  d0 00 34 9c 00 00 00 s    .  e  .  l  .  e  .  c

    ... (the SQL query takes 156 bytes [0x9c] - 78 unicode characters)

    I understand this as following:
    1. 1st 8 bytes (yellow) - packet header, type = 03 = RPC Request
    2. next 4 bytes (green) - proc ID (SP_prep_exec)
    3. next 2 bytes (green too) - options. BTW, according to documentation (2.2.6.4) the options are expected to be a single byte
    4. next 9 bytes (blue) - parameter 1 [name="", flags=1, type=0x26=INTNTYPE, len=4, varlen=4, value=0xffffffff]
    5. I fail to understand the next parameter: name="", flags=0, type=63=NTEXTTYPE. According to 2.2.5.2.3 I expect the TYPE_VARLEN to be USHORTCHARBINLEN (2 bytes), but actually it seems like 4 bytes (as 09 04 d0 00 34 is, probably, a collation info).
      So, 1st question is: why is the length on NTEXTTYPE 4 bytes?
    6. And the 2nd question: how can I continue the parsing? What's the meaning of following ff ff ff ff?

    Thanks in advance,
    Michael

    Wednesday, September 10, 2008 1:31 PM

Answers

  •  

    Michael, Paul,

     

    We have completed our investigation on this inquiry. We agree that the data types NTEXT, TEXT and IMAGE should be documented as LONGLEN_TYPE. 

    The necessary changes will be reflected in a latter release of the [MS-TDS] document. The changes will be similar to the following.

    2.2.5.2.2         Data Type Dependent Data Streams

    Integers

    Data type-dependent integers may be either a BYTELEN, USHORTCHARBINLEN, or LONGLEN in length. This length is dependent on the TYPE_INFO associated with the message. If the data type (for example, FIXEDLENTYPE or VARLENTYPE rule of the TYPE_INFO rule) is of type SSVARIANTTYPE, TEXTTYPE, NTEXTTYPE, IMAGETYPE, the integer length is LONGLEN. If the data type is BIGCHARTYPE, BIGVARCHARTYPE, NCHARTYPE, NVARCHARTYPE, BIGBINARYTYPE, or BIGVARBINARYTYPE, the integer length is USHORTCHARBINLEN. For all other data types, the integer length is BYTELEN.

    2.2.5.4.2         Variable-Length Data Types

          LONGLEN_TYPE     =   IMAGETYPE

                               /

                               NTEXTTYPE

                               /

                               SSVARIANTTYPE

                               /

                               TEXTTYPE

                               /

                                XMLTYPE    ; the length value associated with

                                            these data types is specified

                                            within a LONG

    2.2.5.4.3         Partially Length-Prefixed Data Types

          PARTLENTYPE      =   XMLTYPE

                               /

                               BIGVARCHRTYPE

                               /

                               BIGVARBINTYPE

                               /

                               NVARCHARTYPE

                               /

                               UDTTYPE

     

    Thanks for helping us improve the [MS-TDS] specification.

    Regards,

    Edgar

    • Marked as answer by Chris Mullaney Thursday, November 6, 2008 9:55 PM
    Tuesday, October 14, 2008 10:36 PM
    Moderator

All replies

  • I have alerted our Protocols Support team concerning your questions. One of our team members will contact you soon.
    Thanks!

    Edgar
    Wednesday, September 10, 2008 4:21 PM
    Moderator
  • Hi Michael,

    In order to further assist you on this inquiry, can you provide a network capture (.cap file) of the issue you are observing? 
     
    Which network parser are you using? Wireshark or Microsoft Network Monitor?

    Thanks,

    Edgar
    Wednesday, September 10, 2008 7:43 PM
    Moderator
  • Hi Edgar,

    The capture was recorded with tcpdump, and I've filtered out a single connection using Wireshark. I didn't find a way to attach a file to the message? I could mail it to you (25K), or share it in other way - what's the usual procedure for "attaching" files?

    Thanks
    Thursday, September 11, 2008 6:01 AM
  • Hi Michael,

    I have created a workspace where you can upload your capture for this case:
    Workspace URL: https://sftus.one.microsoft.com/ChooseTransfer.aspx?key=34f86b3a-28c9-49f7-a008-70bd58aa7c04
    Password: E6!M%7miGAC

    Thanks,
    Edgar
    Thursday, September 11, 2008 5:05 PM
    Moderator
  • Not an Microsoft employee, so you will treat my answer accordingly.

    Your point 3: yes, the option flags are 16 bits/2 bytes (and the documentation does appear to be wrong)

    Your points 5 and 6: let me mention a point that helped me understand the structure of the documentation of RPC parameters.

    RPC parameters consist of: name, flags, type, data

    name: is a counted string (B_VARCHAR) - you've got that
    flags: is a single byte - you've got that
    type: is a complex structure starting with a type code but often including a great deal of other information, including some length information
    data: is also a complex structure, which often starts with more/different length information

    To take the simpler INTN example:
    • the type is: INTN, length=4
    • the data is: length=4, <4 byte representation of -1>
    If you wanted to have a 4 byte INTN which is NULL, you do this by:
    • type: INTN, length=4
    • data: length=0 (0 represents NULL, for INTN)
    For NTEXT, the type information includes:
    • 0x63 (NTEXT)
    • length (4 byte value)
    • collation (5 byte value) (if your protocol version includes this)
    and the data value for NTEXT is:
    • length (4 byte value)
    • the actual character data
    Now for NTEXT, NULL values are indicated as a (data) length of -1 (0xffffffff) - a secret the MS-TDS document doesn't include. So the complete interpretation of your second parameter is:

    0x00 - zero length parameter name

    0x00 - flags

    0x63 - NTEXT
    0x00 0x00 0x00 0x00 - NTEXT field length (=0)
    0x09 0x04 0xd0 0x00 0x34 - collation

    0xff 0xff 0xff 0xff - data length - encoding a NULL value

    The third parameter is similar, except here both the field length (0x9c 0x00 0x00 0x00) and data length (0x9c 0x00 0x00 0x00) are non-zero/not NULL, and the data (of length 0x9c) follows.

    The definition of CHARBIN_NULL in 2.2.5.1 indicates, I think, the representation of NULL used for things like NTEXT. The fact that this is 4 bytes long for TEXT, NTEXT, and IMAGE (see 2.2.5.2.3) provides additional confirmation that the length (field length, in my terminology) embedded in the type is a 4 byte value for NTEXT.

    It is my opinion that MS-TDS doesn't adequately document the parameter values: it does tell you how to identify NULL, but not how to identify/translate other data values. You can work it out fairly easily, in fact. It would all be a lot easier if a simple table were included in the documentation, however.

    But the basic structure is always like this: <type info> <data bytes>
    and <type info> is often <type> .. <length> ..
    and <data bytes> is often <length> <actual bytes>,

    so the overall pattern for parameter values is often "<type info> .. <length> .. <length> <actual bytes>" (".." represents interpolated collations etc.). Your INTN is like this, and both the NTEXTs are like this too.

    Best wishes

    Paul Betteridge

    Friday, September 12, 2008 1:51 PM
  • Thanks, Edgar. I've uploaded the file. It's a capture file for single connection, with all empty TCP packets filtered out (SYN, ACKs etc.)
    Friday, September 12, 2008 8:49 PM
  • Paul, thanks a lot for your answer.

    I've missed the part in documentation explaining that 0xffffffff is, actually, a NULL, now I can see it, thanks.

    I still don't understand why is the length of NTEXT represented in 2 bytes. I mean, I can see that in this example you're right, but why should it be 4 bytes? This type was documented as USHORTCHARBINLEN, so I expect it to be 2 bytes - is there a mistake in documentation, or I'm not intepreting it right? (after all I want to write a code that will do this parsing automatically =)

    Thanks again,

    Michael

    Friday, September 12, 2008 8:58 PM
  • Michael

    It is, in my opinion, a mistake in the documentation. NTEXT (and TEXT and IMAGE, although I'm not sure the latter 2 occur in practice) should all be LONGLEN_TYPE.

    You quoted the text in 2.2.5.2.3 which (incorrectly, IMHO) gives the length for NTEXT as 2 bytes. But if you look in 2.2.5.4.2 at the USHORTLEN_TYPE rule, you will see that NTEXT is not listed alongside the other types with a 2 byte length.

    It is listed in 2.2.5.4.3 under the PARTLENTYPE - that's wrong too, I think.

    The documentation is wrong - I'm sure the Microsoft team will confirm that in the end. If you treat NTEXT (and TEXT and IMAGE, although I don't think you will see them in RPC calls) as if it said, in 2.2.5.4.2

    LONGLEN_TYPE = SSVARIANTTYPE / XMLTYPE / NTEXTTYPE / TEXTTYPE / IMAGETYPE;

    you will be able to interpret RPC calls fine.

    I think the other documented data types are documented correctly, but we had a working implementation of all of them before MS-TDS was published, so I have tested all the documentation. There are one or two clarifications or corrections in earlier threads I have opened in this forum.

    Paul
    Friday, September 12, 2008 11:30 PM
  • Again, thanks a lot Paul.

    That would be nice to have the document corrected, but for meanwhile I'll implement the parsing as you suggest.
    Saturday, September 13, 2008 6:21 AM
  • Michael, Paul,
    Thanks for helping us improve the [MS-TDS] specification.
    I apologize we have not been able to respond on this thread more quickly. I have filed a Technical Document Issue against the [MS-TDS] document. I will post the answer once I get confirmation from the product group.
    Thanks
    Edgar
    Thursday, October 9, 2008 11:56 PM
    Moderator
  •  

    Michael, Paul,

     

    We have completed our investigation on this inquiry. We agree that the data types NTEXT, TEXT and IMAGE should be documented as LONGLEN_TYPE. 

    The necessary changes will be reflected in a latter release of the [MS-TDS] document. The changes will be similar to the following.

    2.2.5.2.2         Data Type Dependent Data Streams

    Integers

    Data type-dependent integers may be either a BYTELEN, USHORTCHARBINLEN, or LONGLEN in length. This length is dependent on the TYPE_INFO associated with the message. If the data type (for example, FIXEDLENTYPE or VARLENTYPE rule of the TYPE_INFO rule) is of type SSVARIANTTYPE, TEXTTYPE, NTEXTTYPE, IMAGETYPE, the integer length is LONGLEN. If the data type is BIGCHARTYPE, BIGVARCHARTYPE, NCHARTYPE, NVARCHARTYPE, BIGBINARYTYPE, or BIGVARBINARYTYPE, the integer length is USHORTCHARBINLEN. For all other data types, the integer length is BYTELEN.

    2.2.5.4.2         Variable-Length Data Types

          LONGLEN_TYPE     =   IMAGETYPE

                               /

                               NTEXTTYPE

                               /

                               SSVARIANTTYPE

                               /

                               TEXTTYPE

                               /

                                XMLTYPE    ; the length value associated with

                                            these data types is specified

                                            within a LONG

    2.2.5.4.3         Partially Length-Prefixed Data Types

          PARTLENTYPE      =   XMLTYPE

                               /

                               BIGVARCHRTYPE

                               /

                               BIGVARBINTYPE

                               /

                               NVARCHARTYPE

                               /

                               UDTTYPE

     

    Thanks for helping us improve the [MS-TDS] specification.

    Regards,

    Edgar

    • Marked as answer by Chris Mullaney Thursday, November 6, 2008 9:55 PM
    Tuesday, October 14, 2008 10:36 PM
    Moderator