locked
IKE Main Mode Fails RRS feed

  • Question

  • Hi,

    Using WFP, I've set up a tunnel policy for between a Windows 7 endpoint and proprietary device, using pre-shared key authentication. After initiating traffic from the Windows side, IKE MM succeeds through the key exchange messages. Then, the Windows enpoint sends an IDENITIFCATION payload containing only 40 bytes of encrypted data. More specifically, there is an ISAKMP header identifying "Identification (5)" as the next payload, but the payload is only 40 byes of encrypted data. There is no ISAKMP payload header. Since there is no header, the remote endpoint rejects it, sending a NOTIFICATION of "UNEQUAL-PAYLOAD-LENGTHS". Can anyone tell me why Windows is not putting an ISAKMP payload header on the IDENTIFICATION payload?

    Thanks.

    Wednesday, March 14, 2012 2:25 PM

All replies

  • Can you provide a network capture (e.g. NetMon, Wireshark, etc.)  in addition to a netsh capture ("NetSh.exe WFP Capture Start", repro the issues, "NetSh.exe WFP Capture Stop").  The please post a link to were I can get the resultant files (or send mail to DHarper AT Microsoft DOT com ).

    Also what is the peer device you are communicating with?

    Thanks,


    Dusty Harper [MSFT]
    Microsoft Corporation
    ------------------------------------------------------------
    This posting is provided "AS IS", with NO warranties and confers NO rights
    ------------------------------------------------------------

    Wednesday, March 14, 2012 6:52 PM
    Moderator
  • Dusty,

    Sorry for wasting your time. It turns out that the data following the ISAKMP header is, indeed, supposed to be encrypted. The peer device was throwing out the nonce that Windows was sending because it thought it was too big. This adversely affected the key generation and the ISAKMP payload was not decrypted properly.

    On another note, however, is there a way (callout driver, perhaps?) to see what Windows is sending and receiving in QM messages pre-encryption and post-decryption, respectively?

    Thanks.

    Thursday, March 15, 2012 5:33 PM
  • There are multiple ways to see the data pre / post encryption. You can use an inspection callout sitting below where IPsec processing happens (INBOUND_TRANSPORT_V{4/6} for incoming, and OUTBOUND_TRANSPORT_V{4/6} for outgoing, both in a sublayer weighted lower than IPsec's sublayer (0x8000)) In Windows 8 there is the Trusted intermediary http://msdn.microsoft.com/en-us/library/windows/desktop/hh447438(v=vs.85).aspx. Hope this helps,

    Dusty Harper [MSFT]
    Microsoft Corporation
    ------------------------------------------------------------
    This posting is provided "AS IS", with NO warranties and confers NO rights
    ------------------------------------------------------------

    Monday, March 19, 2012 4:22 AM
    Moderator