locked
AuthenticateAsClient() exception with OpenSSL server

    Question

  • I have a .NET client and an OpenSSL server doing mutual authentication. SslStream.AuthenticateAsClient() throws the exception "A call to SSPI failed, see inner exception" and the inner exception says "The message received was unexpected or badly formatted". An MSDN article says that this can happen if the server's trusted CA list (which is sent to the client during the SSL handshake) doesn't contain the CA that the client cert is using. I'm setting the trusted CA list in the OpenSSL server so that should be valid. The client and server certs were both signed by the same CA which is valid. If I use a .NET server everything works fine. Is there any way to get specific information about what the client doesn't like? Thanks in advance.
    Thursday, April 14, 2011 8:25 PM

Answers

  • The problem was a missing private key for the client certificate. No way to easily determine that though. I had to get people at Microsoft involved. Set this registry key to 7 to turn on all Schannel logging on Win7:

    HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\SecurityProviders\SCHANNEL – EventLogging

    Now the System event log will show Schannel messages. But, the messages may not be very helpful. I received the warning "“…no suitable client certificate could be found" which wasn't really the case. It found the client cert just fine. We also had to run the following commands as admin in a command box to create a special log file, schannel.etl, which Microsoft was able to decipher:

    logman -start schannel -p {37D2C3CD-C5D4-4587-8531-4696C44244C8} 255 3 –ets

    logman -stop schannel –ets

    The log file contained info that told them the real problem was the client certificate's private key needed to be imported into the Windows certifcate store. It's a shame that it took all of that effort to determine the problem.

    Special thanks to the folks at Microsoft for the great help.


    • Marked as answer by TDR99 Thursday, May 12, 2011 1:56 PM
    Tuesday, May 10, 2011 5:54 PM

All replies

  • The problem was a missing private key for the client certificate. No way to easily determine that though. I had to get people at Microsoft involved. Set this registry key to 7 to turn on all Schannel logging on Win7:

    HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\SecurityProviders\SCHANNEL – EventLogging

    Now the System event log will show Schannel messages. But, the messages may not be very helpful. I received the warning "“…no suitable client certificate could be found" which wasn't really the case. It found the client cert just fine. We also had to run the following commands as admin in a command box to create a special log file, schannel.etl, which Microsoft was able to decipher:

    logman -start schannel -p {37D2C3CD-C5D4-4587-8531-4696C44244C8} 255 3 –ets

    logman -stop schannel –ets

    The log file contained info that told them the real problem was the client certificate's private key needed to be imported into the Windows certifcate store. It's a shame that it took all of that effort to determine the problem.

    Special thanks to the folks at Microsoft for the great help.


    • Marked as answer by TDR99 Thursday, May 12, 2011 1:56 PM
    Tuesday, May 10, 2011 5:54 PM
  • Thanks for letting us know these techniques. :-)
    http://www.alanjmcf.me.uk/ Please follow-up in the newsgroup. If I help, mark the question answered
    Thursday, May 12, 2011 9:38 AM