none
Signing Keys RRS feed

  • Question

  • I have some basic questions regarding the calculation of the keys used with SMB signing on the pre SMB2 protocols.

    I've had some success with NTLMv1 authentication by scouring the NLMP spec and with some info on this forum. For NTLMv1 I calculate the session key as indicated in the NLMP spec.

    SessionKey = MD4(NTOWF)  which is essentially the MD4(MD4(unicode password))

    The KXKEY is then calculated as the  HMAC_MD5(sessionkey, Concantenationof(serverchallenge, LMchallengeresponse[0..7]))

    Since Session key exchange is NOT negotiated, the KXKEY is the exportedkey.. etc

    Interestingly, the KXKEY seems to be what is used by the client to sign the messages and no other processing is needed (i.e., the creating a "sign key" from this KXKEY and the magic constant using MD5).  

    Does this seem correct or make sense? From looking at the NLMP spec it seems to imply that two different sign keys are needed? one to verify signs from the client and one to sign the responses? But Like I indicated, once I created the KXKEY and used it to both verify signs from the client and generate them in the responses it worked properly.

    For NTLMv2 I created the Session key based on the info in the NMLP spec as follows:

    Sessionkey = HMAC_MD5(Ntresponsekey,Ntproofstring)

    The spec indicates that, for NTLMv2 the SessionKey IS the KXKEY but i have had no success using it to verify signs from the client with or without the extra step applying the MD5 with the magic constant. What am I missing in this area? Any help would be greatly appreciated. 

    And note, at the moment I am implementing these changes from the server's perspective.

    Wednesday, May 29, 2013 6:58 PM

Answers

  • Sebastian,

    I figured out why NTLMV2 authentication wasn't working for me. Basically the client was sending two challenge responses even when the client is set to send NTLMV2 responses only. Both are calculated using the NTLMV2 algorithm but using different "challenge data".  If I remember correctly one is a stronger hash than the other based upon more SALT data. We authenticated using the first challenge response/ntproof string, BUT the session base key used for signing  is calculated using the second challenge response/nt proof string. Once I made that adjustment things worked properly. The good news is my understanding of the algorithms was correct based upon what I was reading in the various specs. I'm sure there's a blurb somewhere that speaks to this issue of when both responses are present. But maybe I skimmed over it or didn't see it.

    Thanks for your time and all of your help.

    regards,

    Jim

    Monday, June 24, 2013 2:24 PM

All replies

  • Hello Signing Keys

    Thanks for contacting Microsoft Support. A support engineer will be in touch to assist further.

    Regards


    Tarun Chopra | Escalation Engineer | Open Specifications Support Team

    Wednesday, May 29, 2013 8:06 PM
  • Hi Signing Keys,

    I'll be helping you out with this request.

    As soon as I have questions or answers I'll let you know.

    Thanks and regards,


    SEBASTIAN CANEVARI - MSFT Escalation Engineer Protocol Documentation Team

    Wednesday, May 29, 2013 8:16 PM
  • Thanks. I appreciate any info/assistance you can give me. 

    regards,

    Jim

    Thursday, May 30, 2013 1:18 PM
  • Hi Jim,

    Just wanted to drop you a note to let you know that I am working on this and I will let you know as soon as I have news.

    Thanks and regards,


    SEBASTIAN CANEVARI - MSFT Escalation Engineer Protocol Documentation Team

    Wednesday, June 5, 2013 8:54 PM
  • Hi Jim,

     

     

    I was working on my response and a colleague of mine pointed me to a blog entry he wrote a couple years ago.

     

    Please let me know if this makes sense to you. If you have further questions please let me know and we can continue working on this issue.

     

    http://blogs.msdn.com/b/openspecification/archive/2010/04/20/ntlm-keys-and-sundry-stuff.aspx

     

    Thanks and regards,

     

    Sebastian

     

     

     

     


    SEBASTIAN CANEVARI - MSFT Escalation Engineer Protocol Documentation Team

    Monday, June 10, 2013 8:08 PM
  • Hi Sebastan,

    Yes I stumbled over this blog and it added to my questions as to how the sign Key is utilmately derived for Ntlmv2 authentication AND that for NTLMV1 authentication it doesn't appear to be the case that the "Magic signing constant" is applied to create the sign key from the KXKEY. So I'm still puzzled. But I will look at it again and see if there's anything else that might jump out at me.

    For NTLMv1 authentication I create the session key as the MD4(NThash) where the NThash is the MD4(unicode password). The KXKEY is then created by using that session key to HMAC_MD5 the 8 byte sever challenge concantenated with the 8 byte client response. The spec and blog seem to indicate that this KXKEY is concentenated with the "magic signing constant" and MD5 applied to create the sign key. However, in practice the KXKEY itself seems to be the actual key used when verifying signs from the client and signing responses.

    For NTLMV2 authentication I have yet to have success creating the signing key. Like I submitted in my original question. I believe I am creating the "session key" properly and the spec indicates that for NTLMv2 the "session key" IS the KXKEY. So I tried using that key to sign (similar to the NTLMV1 case) and I also tried applying the "magic signing constant" using MD5 but both have been unsuccesful.

    regards,

    Jim

    Tuesday, June 11, 2013 2:36 PM
  • Hi Jim,

    I think I found what might be causing you the issue.

    Newer versions of [MS-NLMP] will show the following changes in the SIGNKEY and SEALKEY sections:

    3.4.5.2   SIGNKEY

    If extended session security is not negotiated (section 2.2.2.5), then no signing keys are available and message signing is not supported.

    If extended session security is negotiated, the signing key is a 128-bit value that is calculated as follows from the random session key and the null-terminated ASCII constants shown.

    -- Input:    

    --   ExportedSessionKey - A randomly generated session key.

    --   NegFlg - Defined in section 3.1.1.

    --   Mode - An enum that defines the local machine performing

         the computation. 

         Mode always takes the value "Client" or "Server.

    --

    -- Output:    

    --   SignKey - The key used for signing messages.

    --

    -- Functions used:

    --   ConcatenationOf(), MD5(), NIL - Defined in Section 6.

    Define SIGNKEY(NegFlg, ExportedSessionKey, Mode) as

    If (NTLMSSP_NEGOTIATE_EXTENDED_SESSIONSECURITY flag is set in NegFlg)

         If (Mode equals "Client")

              Set SignKey to MD5(ConcatenationOf(ExportedSessionKey,

              "session key to client-to-server signing key magic 

              constant"))

         Else

              Set SignKey to MD5(ConcatenationOf(ExportedSessionKey,

              "session key to server-to-client signing key magic

              constant"))

         Endif

    Else

         Set  SignKey to NIL

    Endif

    EndDefine

    3.4.5.3   SEALKEY

    The sealing key function produces an encryption key from the random session key and the null-terminated ASCII constants shown.

    § If extended session security is negotiated, the sealing key has either 40, 56, or 128 bits of entropy stored in a 128-bit value.

    § If extended session security is not negotiated, the sealing key has either 40 or 56 bits of entropy stored in a 64-bit value.

    Note  The MD5 hashes completely overwrite and fill the 64-bit or 128-bit value.

    -- Input:    

    --   ExportedSessionKey - A randomly generated session key.

    --   NegFlg - Defined in section 3.1.1.

    --   Mode - An enum that defines the local machine performing

         the computation.

         Mode always takes the value "Client" or "Server.”

    --

    -- Output:    

    --   SealKey - The key used for sealing messages.

    --

    -- Functions used:

    --   ConcatenationOf(), MD5() - Defined in Section 6.

    Define SEALKEY(NegotiateFlags, ExportedSessionKey, Mode) as

    If (NTLMSSP_NEGOTIATE_EXTENDED_SESSIONSECURITY flag is set in NegFlg)

         If ( NTLMSSP_NEGOTIATE_128 is set in NegFlg)

              Set SealKey to ExportedSessionKey

         ElseIf ( NTLMSSP_NEGOTIATE_56 flag is set in NegFlg)

             Set SealKey to ExportedSessionKey[0..6]

         Else

             Set SealKey to ExportedSessionKey[0..4]

         Endif

         If (Mode equals "Client")

             Set SealKey to MD5(ConcatenationOf(SealKey, "session key to

             client-to-server sealing key magic constant"))

         Else

             Set SealKey to MD5(ConcatenationOf(SealKey, "session key to

             server-to-client sealing key magic constant"))

         Endif

    ElseIf (NTLMSSP_NEGOTIATE_LM_KEY is set in NegFlg)

         If (NTLMSSP_NEGOTIATE_56 flag is set in NegFlg)

              Set SealKey to ConcatenationOf(ExportedSessionKey[0..6], 0xA0)

         Else

              Set SealKey to ConcatenationOf(ExportedSessionKey[0..4], 0xE5,

              0x38, 0xB0)

         EndIf

    Else

         Set SealKey to ExportedSessionKey

    Endif        

    EndDefine

    Note that all instances of RandomSessionKey have been replaced with ExportedSessionKey.

    Please let me know if this answers your questions.

    Thanks and regards,


    SEBASTIAN CANEVARI - MSFT Escalation Engineer Protocol Documentation Team

    Wednesday, June 12, 2013 2:40 PM
  • Thanks. yes I have this Spec. And as I tried to describe. That for

    NTLMV1 it doesn't appear (at least with what I am seeing)  that  the last step is needed. That is, applying the MD5 on the concentenation of the KXKEY with the magic signing constant. The KXKEY itself seems to be the signing key.

    and for NTLMV2 I've tried with and without applying this last step with no success. So I know I am missing something.

    Jim

    Wednesday, June 12, 2013 5:04 PM
  • Just to make sure we are in the same page, the excerpt that I posted is from a version of the document that is newer than the one published at the MSDN site.

    This new  version will come available on the next refresh.

    That being clarified, the changes on the doc were necessary for the calculation to work properly. Please note that on the pseudocode, the value RandomSessionKey has been replaced with ExportedSessionKey for every instance.

    So, I would like to understand if using the keys in the way that I have just stated, still gives you the wrong results.

    If that is the case, then I will need to take this discussion offline to ask you for some debug data and will post the answer on this thread once we reach the solution.

    If we need to do that, then please send an email to my attention to: dochelp@microsoft.com

    Thanks!


    SEBASTIAN CANEVARI - MSFT Escalation Engineer Protocol Documentation Team

    Wednesday, June 12, 2013 5:34 PM
  • Yes the version I was looking at was written as described above and from what I read that since Key exchange is NOT negotiated. The KXKEY is the essentially the random session key which is the exportedsession key. They are all one in the same. And What I have observed is that for

    NTLMV1 authentication the KXKEY calculated as described below appears to be the signing key for both verifying signs from the client and signing responses.

    I calculate the KXKEY as follows:

    the NThash is set to the MD4(Unicode password)

    SESSIONKEY is set to the MD4(NThash) (that is the MD4 of the MD4(Unicode password))

    The KXKEY is set to

    HMAC_MD5(SESSIONKEY, Concant (8 bytes of server challenge, 8 bytes of client challenge response))

    Now here is what is interesting. This result what I am calling the KXKEY appears to be what is used to sign by both client and server.

    So that's question #1. Why isn't the last step applying the MD5 (Concant (KXKEY, magic signing constant))  needed?

     The second part of my question is related to NTLMv2 authentication.

    I calculate the sessionkey as

    SESSIONKEY = HMAC_MD5(Ntresponsekey, Ntproofstring)

    The spec indicates that the SESSIONKEY IS the KXKEY for NTLMv2 authentication. Since Key exchange is NOT negotiated I tried to use this key as the sign key with no luck AND I tried to apply the last step using MD5 on the Concant (KXKEY, magic signing constant) which also didn't work for me. So I am missing something. I just don't know what.

    Note  the magic signing constant I am referring to is that "session key to client-server... " thingy.

    Thursday, June 13, 2013 6:40 PM
  • Hi Jim,

    Please send an email to my attention to dochelp@microsoft.com

    We'll take it from there and I'll post the response to your initial question on this thread once we arrive to the answer.

    Thanks!

    Sebastian


    SEBASTIAN CANEVARI - MSFT Escalation Engineer Protocol Documentation Team

    Thursday, June 13, 2013 8:05 PM
  • Hi Jim,

    As I haven't received your email yet, I'll proceed to post my explanation here.

    Since you've mentioned that you are using SMB, you need to be aware that signing for SMB and SMB2 is based on MS-NLMP but it does not use all of the steps.

    Please take a close look at the following documents and sections as an example of what I am describing:

    [MS-CIFS]

    From 3.2.1.2   Per SMB Connection

    Client.Connection.SigningSessionKey: A variable-length byte array that contains the session key that is used for signing packets, if signing is active.

    If SMB signing is activated on the connection (Client.Connection.IsSigningActive becomes TRUE), the session key from the first non-null, non-guest session is used for signing all traffic on the SMB connection. The Client.Connection.SigningSessionKey is set to one of three values:

    § Empty - If Client.Connection.IsSigningActive is FALSE, no connection signing session key is used.

    § LM Session Key - The LM hash, generated from the user's password using the LMOWFv1() function defined in [MS-NLMP] section 3.3.1.

    § NT Session Key - The NTLM hash, generated from the user's password using the NTOWFv1() function defined in [MS-NLMP] section 3.3.1.

    [MS-SMB]

    From 3.2.5.3   Receiving an SMB_COM_SESSION_SETUP_ANDX Response

    NTLM Authentication

    If the CAP_EXTENDED_SECURITY bit in Client.Connection.ServerCapabilities is not set, then the client processes the response.<97> If the Status field of the response does not contain STATUS_SUCCESS, then the client MUST propagate the error to the application that initiated the authentication. The connection MUST remain open for the client to attempt another authentication.

    If the Status field of the response contains STATUS_SUCCESS, then authentication was successful. The client associates the returned SMB_Header.UID of the response with this user for further requests, as specified in [MS-CIFS]. The Client.Session.AuthenticationState MUST be set to Valid. If the Client.Session.SessionKey is zero, then the client MUST query the authentication package for the 16-byte session key, as specified in [MS-NLMP], and set Client.Session.SessionKey to the returned value. If Client.Session.SessionKey is non-zero, then the client MUST NOT overwrite the existing session key. The client MUST set Client.Session.SessionKeyState to Available.

    Activating Signing

    If authentication has completed successfully, Client.Connection.IsSigningActive is FALSE, and the targeted behavior for this connection is signed according to the description in section 3.2.4.2.3, then the client MUST determine whether signing is required to be activated.

    To determine whether signing is required to be active, the user security context that completed authentication is verified. If the user that authenticated is a guest or is anonymous, then signing MUST NOT be activated. Guest authentication is indicated by bit zero in the Action field of the SMB_COM_SESSION_SETUP_ANDX response being set. Anonymous authentication is indicated by the fact that no credentials are provided.

    If neither of these conditions are true, then the client MUST activate signing as follows:

    § If CAP_EXTENDED_SECURITY is set in Client.Connection.ServerCapabilities, the client MUST use GSS-API to query the session key used in this authentication and store the ExportedSessionKey returned by GSS-API into Client.Connection.SigningSessionKey. The client MUST set Client.Connection.SigningChallengeResponse to NULL.

    § If CAP_EXTENDED_SECURITY is not set in Client.Connection.ServerCapabilities, the client MUST use NTLM to query the session key used in this authentication.

    § For NTLMv1 – the client MUST store SessionBaseKey, returned by the NTOWFv1 function defined in [MS-NLMP] section 3.3.1, into Client.Connection.SigningSessionKey.

    § For NTLMv2 – the client MUST store SessionBaseKey, returned by the NTOWFv2 function defined in [MS-NLMP] section 3.3.2, into Client.Connection.SigningSessionKey.

    The client MUST set Client.Connection.SigningChallengeResponse to the challenge response that is sent in the SMB_COM_SESSION_SETUP_ANDX response.

    From 3.2.5.4   Receiving an SMB_COM_TREE_CONNECT_ANDX Response

    Session Key Protection

    If the response status is STATUS_SUCCESS and the SMB_EXTENDED_SIGNATURE bit is set in the OptionalSupport field of the SMB_COM_TREE_CONNECT_ANDX response, then the client MUST hash the session key of the calling user. This protects the key that is used for signing by making it unavailable to the calling applications.

    The one-way hash MUST be performed on Client.Session.SessionKey that uses the HMAC-MD5 algorithm, as specified in [RFC2104]. The steps are as follows:

    1.  Take the 16-byte user session key from Client.Session.SessionKey.

    § If this is an LM authentication where the session key is only 8 bytes, then zero extend it to 16 bytes.

    § If the session key is more than 16 bytes, then use only the first 16 bytes.

    2.  Calculate the one-way hash as follows:

    CALL hmac_md5( SSKeyHash, 256, session key, session key length, digest )

    SET user session key = digest

    The resulting 16-byte digest is treated as the user's new session key and is returned to the caller who requests it. SSKeyHash is the well-known constant array that is described in section 2.2.2.5.

    After the session key has been hashed, the client MUST place the hash into Client.Session.SessionKey and set Client.Session.SessionKeyState to Available, which allows applications to query the session key. 

    If the TREE_CONNECT_ANDX_EXTENDED_SIGNATURE bit is not set, then the Client.Session.SessionKey is not changed and Client.Session.SessionKeyState MUST be set to Available.

    [MS-SMB2]

    From 3.2.5.3.1   Handling a New Authentication

    § Session.SessionKey MUST be set to the first 16 bytes of the cryptographic key queried from the GSS protocol for this authenticated context. If the cryptographic key is less than 16 bytes, it is right-padded with zero bytes. For information about how this is calculated for Kerberos authentication using Generic Security Service Application Programming Interface (GSS-API), see [MS-KILE] section 3.1.1.2. For information about how this is calculated for NTLM authentication using GSS-API, see [MS-NLMP] section 3.1.5.1.

    This information, together with the changes to MS-NLMP that I have described earlier, should help you in achieving your goal.

    Please let me know if you need further assistance.

    Thanks and regards,


    SEBASTIAN CANEVARI - MSFT Escalation Engineer Protocol Documentation Team

    Friday, June 14, 2013 4:31 PM
  • Actually I think I said I have pre smb2. In reality its mostly the CIFS protocol that is implemented. We do negotiate CAP_EXTENDED_SECURITY to get the SNEGO wrapping but like I said for NTLMV1 authentication the signing key seems to be calculated as

    Session base key is set to MD4 (NTOWF) where NTOWF is the MD4(unicode password)

    KXKEY is set to HMAC(Session Basekey, Concantenationof(server challenge, lmchallengeresponse[0..7]))

    Since key exchange is not negotiated the KXKEY is the exportedsessionkey AND seems to be the actual signing key for both directions. (i.e., veriyfing signs and signing response).

    The NTLMSP spec and the blog you referred me to seems to indicate that the there is another step to calculate the signing key. That is

    SIGNKEY(Negflag, ExportedSessionkey, "client")

    SIGNKEY(Negflg,Exportedsessionkey, "server")

    these interfaces are defined to apply MD5 over the concentation of the exportedsessionkey concentatentated with the "magic constant". In my testing it doesn't seem that this last step is needed for NTLMv1 and the KXKEY IS the actual signing key. That puzzled me. And I was wondering why that is.

    For NTLMV2 I calculate the keys as per the NLMP spec and as indicated in the blog as

    Sessionbasekey is set to HMAC_MD5(ResponseKeyNT, NTproofstring)

    where responsekeynt as defined in the NMLP spec

    for NTLMV2 authentication the KXKEY is defined to be the same as the sessionbasekey. I've tried using this as the signing key AND I've tried applying the SIGNKEY interface using this KXKEY as the exportedsessionkey with no success.

    Note domain login and/or kerberos is not in effect.

    Monday, June 17, 2013 6:37 PM
  • Hi Jim,

    As I posted above, CIFS (nor SMB nor SMB2 for that matter) does not use the full set of steps to calculate the signing key as specified in NLMP. It just uses part of it:

    [MS-CIFS]

    From 3.2.1.2   Per SMB Connection

    Client.Connection.SigningSessionKey: A variable-length byte array that contains the session key that is used for signing packets, if signing is active.

    If SMB signing is activated on the connection (Client.Connection.IsSigningActive becomes TRUE), the session key from the first non-null, non-guest session is used for signing all traffic on the SMB connection. The Client.Connection.SigningSessionKey is set to one of three values:

    § Empty - If Client.Connection.IsSigningActive is FALSE, no connection signing session key is used.

    § LM Session Key - The LM hash, generated from the user's password using the LMOWFv1() function defined in [MS-NLMP] section 3.3.1.

    § NT Session Key - The NTLM hash, generated from the user's password using the NTOWFv1() function defined in [MS-NLMP] section 3.3.1.

    The signkey function is not applied but only the algorithm on NLMP 3.3.1

    This ends up being:

                     Set SessionBaseKey to MD4(NTOWF)

    With regards to NTLM v2:

                    Set SessionBaseKey to HMAC_MD5(ResponseKeyNT, NTProofStr)

    If this is not working for you, I need you to please send a network capture and the corresponding user password to my attention at dochelp@microsoft.com

    Thanks!


    SEBASTIAN CANEVARI - MSFT Escalation Engineer Protocol Documentation Team


    Monday, June 17, 2013 10:16 PM
  • Ok thanks. For CIFS, You've confirmed my speculation that the SIGNKEY algorithm is not applied. Which is what I am seeing for the NTLMv1 authentication which is working. Can you confirm that for NTLMV2 that the signing key  is the sessionbasekey as described above? That is once we calculate the session base key via HMAC_MD5(ResponseKeyNT, NTProofStr) that this should be the signing key as well?  If that's the case, then I want to dig into the implementation further and if I still have questions I will email as you requested with wireshark traces.

    Thanks for your help !

    Jim

    Wednesday, June 19, 2013 12:23 PM
  • Hi Jim,

    Thanks for your reply.

    Yes, that is the way we calculate the keys in NTLMv2.

    Now, something crossed my mind that I want to make sure that we are in the same page about.

    It is a common misconception to understand that the selection between ntlmv1 and ntlmv2 is controlled by the NTLMSSP_NEGOTIATE_NTLM2 flag when in fact, it is controlled by the LMCompatibilityLevel policy.

    As long as you are in fact setting the version through the policy, then we are good but if you are assuming that NTLMv2 is being selected because of the flag setting, then I need to tell you to try your implementation of ntlmv2 AFTER changing the policy value (through policy or through registry).

    In case you need more details, these links can give you more insight:

    http://technet.microsoft.com/en-us/magazine/2006.08.securitywatch.aspx

    http://technet.microsoft.com/en-us/library/cc960646.aspx

    Either way, feel free to contact us through dochelp@microsoft.com if you have further questions.

    Thanks and regards,


    SEBASTIAN CANEVARI - MSFT Escalation Engineer Protocol Documentation Team


    Thursday, June 20, 2013 4:01 PM
  • Sebastian,

    I figured out why NTLMV2 authentication wasn't working for me. Basically the client was sending two challenge responses even when the client is set to send NTLMV2 responses only. Both are calculated using the NTLMV2 algorithm but using different "challenge data".  If I remember correctly one is a stronger hash than the other based upon more SALT data. We authenticated using the first challenge response/ntproof string, BUT the session base key used for signing  is calculated using the second challenge response/nt proof string. Once I made that adjustment things worked properly. The good news is my understanding of the algorithms was correct based upon what I was reading in the various specs. I'm sure there's a blurb somewhere that speaks to this issue of when both responses are present. But maybe I skimmed over it or didn't see it.

    Thanks for your time and all of your help.

    regards,

    Jim

    Monday, June 24, 2013 2:24 PM