none
MS-OXPSVAL - v1.02 - documentation problems RRS feed

  • Question

  • I'm having hard time trying to implement postmark verification algorithm described in this document. There are several things that are not clear to me.

    1. Son-of-SHA1 alg is defined the same as SHA1, with inclusion of one XOR operation g() (page 11), and different constants (page 13). But there is another known Son-of-SHA1 implementation, described in CSRI document (http://download.microsoft.com/download/7/6/b/76b1a9e6-e240-4678-bcc7-fa2d4c1142ea/csri.pdf) that mentions only XOR operation, but not constants. Further, CSRI flavor provides test data, which I have been able to verify, in contrast to MS-OXPSVAL (for which I was not able to verify my implementation). So, the question here is, what is correct description for Son-of-SHA1 protocol used in MS Outlook, and are there test data available for checking Son-of-SHA1 implementation?

    2. Page 10 of MS-OXPSVAL mentions following as second condition for valid solution:
    "The last 12 bits of each of the documents δ are the same (the particular 12-bit suffix 

    value shared by these documents does not matter). "


    However, given example data at end of that document (page 15): X-CR-HashedPuzzle: BjHi CbbP CsE4 DoWO EhAv FJE7 FMx3 FOJO FjsQ HDPJ IFAE IRyJ I5E3 I+BV KBb7 L+gd


    I was unable to recognize 12bit common suffix in these 16 solutions? Is documentation incorrect, or my understanding of "the last 12 bits of each.."?


    Thanks,

    Vladimir Zidar




    Saturday, October 25, 2008 12:29 PM

Answers

  • Vladimir,

    Question #1 the CSRI document is out-dated and doesn’t accurately represent the algorithm implemented in Exchange 2007 and Outlook 2007. The new constants are necessary to produce postmark valid to Outlook  and Exchange.

    And for question #2  the document is wrong here. It is not the last 12 bits of each δ are the same. It should be “the last 12 bits of each H(δ o H(NWS(D))) are the same”.  A Technical Document Issue (TDI) has been filed to correct the document.

    Also note that the the pseudo algorithm should read on page 11:

    Solution = 0
    While(true)

        Hash = H(concatinate(Solution, H(NWS(D)))) 
        If Verify(Solution, Puzzle) succeeds 
        { 
            Remember this Solution and Hash 
            If we have 16 Solutions where the last 12 bits of their corresponding Hash are the same 
            { 
                Return these 16 Solutions 
            } 
        } 
        Solution ++
    }

    Developer Consultant
    Tuesday, November 4, 2008 5:38 AM
    Moderator

All replies

  • Good morning Vladimir. I have alerted our Protocols Support team concerning your questions regarding MS-OXPSVAL. One of our team members will be in contact with you soon.

    Regards,
    Richard Guthrie

    Escalation Engineer
    Saturday, October 25, 2008 5:04 PM
  • Hi Vladimir,

    I have filed a Technical Document Question (TDQ) asking for clarfication and response to your question around the Son-of-SHA1 algorithm. 

    Thank you for your question please don't hesitate to ask additional question about the documentation.
    Developer Consultant
    Tuesday, October 28, 2008 4:47 PM
    Moderator
  • Hi Richard, hi Tom,

    is there a place where I can follow up resolution of that TDQ?

    Thanks,
    Vladimir

    Tuesday, October 28, 2008 11:21 PM
  • I will post a follow-up response  once I get feedback from development to your questions. 


    Developer Consultant
    Wednesday, October 29, 2008 8:55 PM
    Moderator
  • Vladimir,

    Question #1 the CSRI document is out-dated and doesn’t accurately represent the algorithm implemented in Exchange 2007 and Outlook 2007. The new constants are necessary to produce postmark valid to Outlook  and Exchange.

    And for question #2  the document is wrong here. It is not the last 12 bits of each δ are the same. It should be “the last 12 bits of each H(δ o H(NWS(D))) are the same”.  A Technical Document Issue (TDI) has been filed to correct the document.

    Also note that the the pseudo algorithm should read on page 11:

    Solution = 0
    While(true)

        Hash = H(concatinate(Solution, H(NWS(D)))) 
        If Verify(Solution, Puzzle) succeeds 
        { 
            Remember this Solution and Hash 
            If we have 16 Solutions where the last 12 bits of their corresponding Hash are the same 
            { 
                Return these 16 Solutions 
            } 
        } 
        Solution ++
    }

    Developer Consultant
    Tuesday, November 4, 2008 5:38 AM
    Moderator
  • Tom,

    this clarifies some issues, but I am still not able to verify puzzles. It could easily boil down to my implementation of Son-Of-Sha1 alg being broken.

    Can I ask for test data for sosha1 alg, in the similar way they are provided for regular Sha1?

    Thanks,
    Vladimir

    Thursday, November 6, 2008 12:40 PM
  • Will do.


    Developer Consultant
    Friday, November 7, 2008 6:15 AM
    Moderator
  • Vladimir,

    Sorry I was hoping to have gotten the data by now.  I am still waiting for a response from development. 
    Developer Consultant
    Saturday, November 15, 2008 5:48 AM
    Moderator
  • Tom,

    I really appreciate your efforts!

    Thanks,
    Vladimir

    Sunday, November 16, 2008 5:04 AM
  • Vladimir,

    I was hoping to provide you the test data but the feedback that I received is this is outside of scope of the protocol documentation. 


    Developer Consultant
    Monday, November 24, 2008 5:22 AM
    Moderator
  • So the official response is that test data is impossible to obtain for Microsoft SonOfSha1 variation, which as a consequence makes postmark verification extremely hard to implement?

    Is there any known 3rd party implementation available? Either open or closed source, just for record?

    Thanks,
    Vladimir
    NetManagement.net

    Wednesday, November 26, 2008 6:47 PM
  • Good news is I got development to supply me the test data for Outlook 2007.

    Here is the original table:

    Constants: 0x5A827999 ,  0x6ED9EBA1 , 0x8F1BBCDC, 0xCA62C1D6

     

    the string "abc"

    EBF90F28  917D0F67  A0994009  290FAC95  A0B32507

     

    the string "abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq"

    76BD1E7E  CB11C355  EB9AC016  FC5C2D29  9C9617DB

     

    a string consisting of 1,000,000 a's

    37362E7A  6C3339B4  3FC135CC  0E2E9707  9C2E53D2

     

    an empty string

    C60F381D  0342B6DE  22C66FE1  C37968BC  D7D97C08

    And here is the table with Outlook 2007 constants:

    Constants: 0x041D0411,  0x416C6578, 0xA116F5B6, 0x404B2429

     

    the string "abc"

    FA12E295  9DB79C97  25338C0F  D4DE3E01  78C286BD

     

    the string "abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq"

    48F6CE9F  DCF53F40  89200091  ED9739E1  7D73D975

     

    a string consisting of 1,000,000 a's

    57338A4C  C33E70D4  3A3D3AD7  E93C85ED  E6996CCD

     

    an empty string

    7A790886  F5044A7B  DA812BA8  BFC286C4  F51E7B34

     


    Developer Consultant
    Friday, November 28, 2008 4:26 PM
    Moderator
  • Tom, thanks!

    I used these to verify my sosha1 implementation, and to my surprise, it was 100% correct. Unfortunately, that means that something else is still misinterpreted or otherwise wrong here..

    #include <string> 
    #include "SonOfSHA1.h" 
     
    int main() 
    // document without white space NWS(D) 
      const char *document = "1;dQBzAGUAcgAxAEAAZQB4AGEAbQBwAGwAZQAuAGMAbwBtAA==;Sosha1_v1;7;{d04b23f4-b443-453a-abc6-3d08b5a9a334};cwBlAG4AZABlAHIAQABlAHgAYQBtAHAAbABlAC4AYwBvAG0A;Tue,01Jan200808:00:00GMT;SABlAGwAbABvAA=="
     
    // H(NWS(D)) 
      unsigned int dm[5]; 
       
      SonOfSHA1 sosha11; 
      sosha11.Reset(); 
      sosha11.Input(document, strlen(document)); 
      sosha11.Result(dm); 
     
     
    //  const char *solution1 = "BjHi"; 
      const unsigned char solution1_binary[3] = { 0x06, 0x31, 0xe2 }; 
       
       
      // concatenate(solution, H(NWS(D))) 
      SonOfSHA1 sosha12; 
      sosha12.Reset(); 
      sosha12.Input(solution1_binary, 3); 
      sosha12.Input((unsigned char *) dm, sizeof(dm)); 
      // and make hash H(concat(solution, H(NWS(D)))) 
       
      unsigned int h[5]; 
      sosha12.Result(h); 
       
      // so now last 12 bits of h should be..? 
       
      printf("0x%08x\n", h[0]); 
      printf("0x%08x\n", h[1]); 
      printf("0x%08x\n", h[2]); 
      printf("0x%08x\n", h[3]); 
      printf("0x%08x\n", h[4]); 
       

    And what I get for output is this:

    0xba032717
    0x00257803
    0x1ec08a09
    0x5324afed
    0xfc0f832f

    It is to my understanding that 12 same bits would mean that at least one of the individual bytes would have all bits the same.. which I fail to see here..

    Please, I am starting to be desperate on this, as there are some people up the food chain that are breathing into my neck to get this done...

    Tuesday, December 16, 2008 7:22 PM
  • Hi Vladimir,

    Sorry I missed your follow-up as I have been on holiday.  Were you able to resolve this?

    I passed your feedback to the developers who gave me the test data above.
    Developer Consultant
    Thursday, January 8, 2009 8:12 PM
    Moderator
  • Vladimir,   Where you able to figure this out? 


    Developer Consultant
    Thursday, January 15, 2009 7:19 PM
    Moderator
  • Marking as Answered since there has been no response.


    Developer Consultant
    Saturday, January 17, 2009 1:14 AM
    Moderator
  • Sorry Tom, I was at the other side of the world too.

    Anyway, couldn't figure out what is wrong with my example.

    Can you please get in touch with developers and to find out why my example is not working. The alg is 100% correct, and if the docs are correct too, then I must have been doing something completely wrong.

    Thanks



    Thursday, January 22, 2009 10:09 PM
  • Looks like you are trying to compare the last 12 bits of h[0] through h[4].  According to the spec,  you first must compute an h for each solution, and then verify that each h for each solution that has the same last 12 bits.

    You should have a DWORD h[16][5], and compare the last 12 bits of h[0] through h[15].


    Developer Consultant
    Thursday, January 22, 2009 11:28 PM
    Moderator