Skip to main content

 none
IPSec Transport Mode with certificate RRS feed

  • Question

  • I have taken the IPSec transport mode sample and tried to modify it so that Kerberos or NTLM will not be used, but only certificates can be used for authentication. Unfortunately, with only slight modifications to the code, I cannot get it to work. Is there some EKU or other attribute of a certificate that I should ensure I have to make this work?

    Note that I can use the same certificates in configuring a server-to-server IPSec rule in the Adv Firewall UI. Also note that i can get preshared key and other auth methods to work in my IPSec transport mode code. I just cannot take those same certificates and get them to work with my code. I have tried AuthIP as well as IKE. I have tried IKEEXT_SSL as well as IKEEXT_CERTIFICATE. I have tried adding these EKUs to my certs:

    Client Authentication (1.3.6.1.5.5.7.3.2)
    Server Authentication (1.3.6.1.5.5.7.3.1)
    IP security IKE intermediate (1.3.6.1.5.5.8.2.2)

    I have tried IKE logging, and without the wfp.tmf file I can only make out the same error:

    F a i l u r e   e r r o r   c o d e : 0 x 0 0 0 0 3 5 e 9 
             I K E   a u t h e n t i c a t i o n   c r e d e n t i a l s   a r e   u n a c c e p t a b l e

    What else can I do to debug this?

    I am trying Server 2012 and Windows 10.


    Wednesday, July 31, 2019 2:06 AM

Answers

  • Hello PhilipHamer,

    Maybe you can refer to the following possible reasons can result in the error 0x35e9 (13801):

    Error 13801 occurs on the client when:

    • The certificate is expired.

    • The trusted root for the certificate is not present on the client.

    • The subject name of the certificate does not match the remote computer.

    • The certificate does not have the required Enhanced Key Usage (EKU) values assigned.

    And I have some information need to confirm with you:

    • Can you show the link of IPSec transport mode sample?
    • What's kind of modifications have you made to to the sample?
    • Are you configuring a IPSec rule between a server and a client? Where do you get this error, on client or on server?

    Best regards,

    Rita


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    • Marked as answer by PhilipHamer Thursday, August 1, 2019 12:48 PM
    Wednesday, July 31, 2019 8:20 AM

All replies

  • Hello PhilipHamer,

    Maybe you can refer to the following possible reasons can result in the error 0x35e9 (13801):

    Error 13801 occurs on the client when:

    • The certificate is expired.

    • The trusted root for the certificate is not present on the client.

    • The subject name of the certificate does not match the remote computer.

    • The certificate does not have the required Enhanced Key Usage (EKU) values assigned.

    And I have some information need to confirm with you:

    • Can you show the link of IPSec transport mode sample?
    • What's kind of modifications have you made to to the sample?
    • Are you configuring a IPSec rule between a server and a client? Where do you get this error, on client or on server?

    Best regards,

    Rita


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    • Marked as answer by PhilipHamer Thursday, August 1, 2019 12:48 PM
    Wednesday, July 31, 2019 8:20 AM
  • Thank you for your response, Rita. As it turns out, your requesting to know what modifications I made to the code, made me reexamine things. And now I understand that with AuthIP, I needed to remove the extended mode authentication options. (I had read that extended mode auth was optional, but I interpreted that to mean that it does not need to succeed if main mode succeeds. But it seems it just means that you can set emPolicy=NULL.)

    So that being said, I got things to work with the IKEEXT_SSL main mode auth and AuthIP. However, I still cannot get IKEEXT_CERTIFICATE with IKE (v1 or v2) to work.

    If anyone has any insights on that, please share. Otherwise, I will move on with AuthIP.

    Thanks,
    Philip

    Wednesday, July 31, 2019 2:55 PM