none
Microsoft DIRECT2 algorithm question. RRS feed

  • Question

  • While decompressing, I found sometime the (offset, length) pair giving me value where offset is more that my current output buffer offset. How taht is possible?

    As an example, lets assume,

    - I consumed 20 bytes from my compress data

    - then I got a pair (251, 5)

    - surprisingly at the point I can go maximum of 20 bytes

    How I should handle this? Currently I am skipping such input. Any reason, please explain.

     

    Friday, August 20, 2010 1:09 AM

Answers

  • Tapan,

    I’m closing this discussion due to lack of response.  Please feel free to repost to the forum if you still have questions.

    Best regards,

    Tom Jebo

    Escalation Engineer

    Microsoft Open Specifications

    • Marked as answer by Tom Jebo MSFT Friday, October 15, 2010 6:37 PM
    Friday, October 15, 2010 6:37 PM

All replies

  • ktapan,

    I'm sorry I don't know the answer to your question. However it would probably help the Microsoft team if you can confirm which specification you are using (e.g. MS-OXCRPC), describe the test environment (e.g. which server or client tool and which version you are testing against) and provide a sample of the data that you are decoding that shows the problem.

    Brad

    Friday, August 20, 2010 1:31 AM
  • HI Brad,

              Thanks for the response. Yes I am using  MS-OXCRPC document dated 21May 2010. I save some MAPI payload (after extracting MSRPC frame). The RPC_HEADER_EXT indicate the payload is compressed in flags. I am trying to up compress the pay load to parse the ROPS. I wrote my decompression function as instructed in MS-OXCRPC document. Here ir is what see;

    - I got 32 bits or 4 bytes (mask in big endian format).

    - The bit in the mask is set to 1

    - so I am reading next 2 bytes to get offset and length pair. I believe first 13 bytes are the offset and last 3 bits are length

    - yes for length more than 6 ( I mean 7 and more) we need to read shared nibble)

    - to get offset here is what I am doing;

    -  get next two bytes (as indicated above)

    - divide it by 8 gives me offset

    - mod 8 will give me length (1st level). Yes if this length is 7 we need to dig more.

    - After having all this done, I see the offset value (from where the data to be copied is more that output_index

                  -------------------------^

                                                  |     current position

    |-------offset value ---------------|

     As shown above the offset value is more that the currently written data. Why this is happening.

    I will try to upload my data file tomorrow.

     

    Thanks,

    -Tapan

     

     

     

    Friday, August 20, 2010 4:39 AM
  • Tapan,

    Thanks for your inquiry regarding the Direct2 algorithm usage in MS-OXCRPC. One of our engineers will follow-up with you shortly.

    Regards,

    Edgar

    Friday, August 20, 2010 2:50 PM
    Moderator
  • Hi Tapan,

    I'll be working with you on this issue.  Thanks for the details of your issue.  Your algorithm description looks generally correct to me (per MS-OXCRPC 3.1.7.2.2.4 Match Length).  Please send me your data so I can verify what you're seeing.  Just send the file (zipped) to "dochelp" at Microsoft.com and reference my name and the link to this forum post. 

    Best regards,
    Tom Jebo
    Escalation Engineer
    Microsoft Open Specifications

    Monday, August 23, 2010 3:21 PM
  • Hi Tom,

             Thanks for taking initiative. I already send you the data in attachment with an e-mail to DocHelp@Microsoft.com. I also explane how to read my sample data in the e-mail. In case of any question please let me know.

    Thanks,

    -Tapan

     

    Monday, August 23, 2010 10:01 PM
  • Tapan,

    I've sent you a couple of emails but since you're not responding, I'll post my questions here.  Please check your email for my detailed findings.  Also, can you answer these questions about your sample data:

    1)      Why is the length 0x00000093?  If I assume that the 4 byte length is included and go to offset 0x93, the byte there has only two length bytes 0x0100 before the “kundu” string.   If I don’t assume the 4 byte length is included and go to offset 0x97, then it overlaps into the “kundu” string.

    2)      I don’t see anything that appears to be 4 byte lengths after each “kundu” string.  Are there supposed to be?  You said the pattern repeats.

    3)      The bitmasks (alleged) that come after the “kundu” string don’t look valid or aren’t working.   Could you copy the data directly from the stream starting with the RPC_HEADER_EXT and including the data?

     

    Best regards,

    Tom Jebo

    Escalation Engineer

    Microsoft Open Specifications

     

    Friday, September 10, 2010 3:38 PM
  • Hi Tom,

               Thanks for your response in the forum.  I didn't receive any e-mail from you. I send you couple of reminder in between using your direct email address.

    Here is how to read the data file I provided. The file starts with 4 bytes (unsigned int) length, followed by length number of bytes. The string "kundu" is a terminator string for each data pattern. In short here is the format of the data file

       <length1> <data of length1 bytes > "kundu" <length2> <data of length2> "kundu" and so on

    To use decompression function use following steps;

     - open the given file with data ( I provided)

     - read 4 bytes of length (assume its value is length1)

     - read length1 bytes into a buffer

    - call decompression function to decompresses the buffer

    - skip next 5 bytes of data (which is the string "kundu")

    Hope this will help you.

    -Tapan

     

    Sunday, September 19, 2010 3:36 AM
  • Hi Tapan,

    As I mentioned in my last response, the length and "kundu" string are not matching up correctly.  If I copy 0x93 bytes of data, I get some of the "kundu" string.  I'm not confident that I've got valid compressed data to process.  That's why I asked you for the data directly from the stream starting with RPC_HEADER_EXT and including the data.   If the file you sent me was not valid, please send a valid one to dochelp at microsoft.com. 

    Best regards,
    Tom Jebo
    Escalation Engineer
    Microsoft Open Specifications

     

    Monday, September 20, 2010 3:59 PM
  • Tapan,

    I’m closing this discussion due to lack of response.  Please feel free to repost to the forum if you still have questions.

    Best regards,

    Tom Jebo

    Escalation Engineer

    Microsoft Open Specifications

    • Marked as answer by Tom Jebo MSFT Friday, October 15, 2010 6:37 PM
    Friday, October 15, 2010 6:37 PM