locked
CryptographicException: "Key does not exist" when attempting to connect to remote service RRS feed

  • Question

  • I am working on creating a self hosted WCF service and client (.NET 4.0).  This is not going through IIS.  This system will be running on an intranet without an external connection to the internet.

    Our bindings look like (this one specifically the client):

          // Setup binding
          binding = new NetTcpBinding(SecurityMode.TransportWithMessageCredential, true);
          binding.OpenTimeout = new TimeSpan(0, 0, 5);
          binding.Security.Transport.ClientCredentialType = TcpClientCredentialType.Certificate;
          binding.Security.Message.ClientCredentialType = MessageCredentialType.Certificate;
    
          // Setup remote endpoint
          EndpointAddressBuilder addressBuilder = new EndpointAddressBuilder();
          addressBuilder.Uri = remoteServiceUri;
          addressBuilder.Identity = EndpointIdentity.CreateX509CertificateIdentity(remoteCertificate);
          endpointAddress = addressBuilder.ToEndpointAddress();
    
          // Load client credentials
          clientCertificate = localPfxCertificate;
    
    ...
    
            RcaClientImpl clientImpl = new RcaClientImpl(this, binding, endpointAddress);
            clientImpl.ChannelFactory.Credentials.ServiceCertificate.Authentication.CertificateValidationMode =
                X509CertificateValidationMode.None;
            clientImpl.ClientCredentials.ClientCertificate.Certificate = clientCertificate;
    

    Where:
    - remoteCertificate is a X509Certificate2 containing the public key of the remote service we are connecting to.
    - clientCertificate is a X509Certificate2 containing the private key identifying the local client

    However, I am running into a problem with the clientCertificate (I think).  If I use makecert/pvk2pfx to create the client certificate, ex:
    makecert -sv test.pvk -n "CN=Test" test.cer
    pvk2pfx -pvk test.pvk -spc test.cer -pfx test.pfx

    And I then load the generated .pfx file, I get an expected error (that is, the service rejects the process because it does not know about the newly generated certificate.  And that is OK - it is separately added to the service).

    We were looking for a way to programmatically generate the certificates instead of using those command line tools, and came across the BouncyCastle library.  Notably, based on the example here (http://www.fkollmann.de/v2/post/Creating-certificates-using-BouncyCastle.aspx) in step 2.  At the end, we convert the BouncyCastle defined certificate into the native .NET X509Certificate2 and add the private key to it.

          RsaKeyPairGenerator kpGen = new RsaKeyPairGenerator();
          kpGen.Init(new KeyGenerationParameters(new SecureRandom(new CryptoApiRandomGenerator()), 1024));
    
          AsymmetricCipherKeyPair keyPair = kpGen.GenerateKeyPair();
          X509V3CertificateGenerator gen = new X509V3CertificateGenerator();
          X509Name certificateName = new X509Name("CN=" + rcaMain.centerId);
          BigInteger serialNumber = BigInteger.ProbablePrime(120, new Random());
          gen.SetSerialNumber(serialNumber);
          gen.SetSubjectDN(certificateName);
          gen.SetIssuerDN(certificateName);
          gen.SetNotAfter(DateTime.Now.AddYears(100));
          gen.SetNotBefore(DateTime.Now.AddDays(-1));
          gen.SetSignatureAlgorithm("SHA1WITHRSA");
          gen.SetPublicKey(keyPair.Public);
    
          gen.AddExtension(X509Extensions.AuthorityKeyIdentifier.Id, false,
                   new AuthorityKeyIdentifier(
                     SubjectPublicKeyInfoFactory.CreateSubjectPublicKeyInfo(keyPair.Public),
                     new GeneralNames(new GeneralName(certificateName)), serialNumber));
    
          X509Certificate newCertificate = gen.Generate(keyPair.Private);
    
          // Having generated the new certificate, convert it to the same type used natively by .NET
          System.Security.Cryptography.X509Certificates.X509Certificate dotnetCertificate = DotNetUtilities.ToX509Certificate(newCertificate);
          X509Certificate2 dotnetCertificate2 = new X509Certificate2(dotnetCertificate.GetRawCertData(), "");
          dotnetCertificate2.PrivateKey = DotNetUtilities.ToRSA((RsaPrivateCrtKeyParameters)keyPair.Private);
    

    However, upon then calling .Open() on the client, I get the following exception (sorry, it's long):
    System.Security.Cryptography.CryptographicException: Key does not exist.


    Server stack trace:
       at System.Security.Cryptography.Utils.GetKeyPairHelper(CspAlgorithmType keyType, CspParameters parameters, Boolean randomKeyContainer, Int32 dwKeySize, SafeProvHandle& safeProvHandle, SafeKeyHandle& safeKeyHandle)
       at System.Security.Cryptography.RSACryptoServiceProvider.GetKeyPair()
       at System.Security.Cryptography.RSACryptoServiceProvider.SignHash(Byte[] rgbHash, String str)
       at System.IdentityModel.SignedXml.ComputeSignature(HashAlgorithm hash, AsymmetricSignatureFormatter formatter, String signatureMethod)
       at System.IdentityModel.SignedXml.ComputeSignature(SecurityKey signingKey)
       at System.ServiceModel.Security.WSSecurityOneDotZeroSendSecurityHeader.CompletePrimarySignatureCore(SendSecurityHeaderElement[] signatureConfirmations, SecurityToken[] signedEndorsingTokens, SecurityToken[] signedTokens, SendSecurityHeaderElement[] basicTokens)
       at System.ServiceModel.Security.WSSecurityOneDotZeroSendSecurityHeader.CreateSupportingSignature(SecurityToken token, SecurityKeyIdentifier identifier)
       at System.ServiceModel.Security.SendSecurityHeader.SignWithSupportingToken(SecurityToken token, SecurityKeyIdentifierClause identifierClause)
       at System.ServiceModel.Security.SendSecurityHeader.SignWithSupportingTokens()
       at System.ServiceModel.Security.SendSecurityHeader.CompleteSecurityApplication()
       at System.ServiceModel.Security.SecurityAppliedMessage.OnWriteMessage(XmlDictionaryWriter writer)
       at System.ServiceModel.Channels.BufferedMessageWriter.WriteMessage(Message message, BufferManager bufferManager, Int32 initialOffset, Int32 maxSizeQuota)
       at System.ServiceModel.Channels.BinaryMessageEncoderFactory.BinaryMessageEncoder.WriteMessage(Message message, Int32 maxMessageSize, BufferManager bufferManager, Int32 messageOffset)
       at System.ServiceModel.Channels.FramingDuplexSessionChannel.EncodeMessage(Message message)
       at System.ServiceModel.Channels.FramingDuplexSessionChannel.OnSend(Message message, TimeSpan timeout)
       at System.ServiceModel.Channels.OutputChannel.Send(Message message, TimeSpan timeout)
       at System.ServiceModel.Dispatcher.DuplexChannelBinder.Request(Message message, TimeSpan timeout)
       at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
       at System.ServiceModel.Channels.ServiceChannel.Request(Message message, TimeSpan timeout)

    Exception rethrown at [0]:
       at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
       at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
       at System.ServiceModel.Channels.IRequestChannel.Request(Message message, TimeSpan timeout)
       at System.ServiceModel.Channels.SecurityChannelFactory`1.SecurityRequestChannel.Request(Message message, TimeSpan timeout)
       at System.ServiceModel.Security.SecuritySessionSecurityTokenProvider.DoOperation(SecuritySessionOperation operation, EndpointAddress target, Uri via, SecurityToken currentToken, TimeSpan timeout)
       at System.ServiceModel.Security.SecuritySessionSecurityTokenProvider.GetTokenCore(TimeSpan timeout)
       at System.IdentityModel.Selectors.SecurityTokenProvider.GetToken(TimeSpan timeout)
       at System.ServiceModel.Security.SecuritySessionClientSettings`1.ClientSecuritySessionChannel.OnOpen(TimeSpan timeout)
       at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
       at System.ServiceModel.Channels.ReliableChannelBinder`1.ChannelSynchronizer.SyncWaiter.TryGetChannel()
       at System.ServiceModel.Channels.ReliableChannelBinder`1.ChannelSynchronizer.SyncWaiter.TryWait(TChannel& channel)
       at System.ServiceModel.Channels.ReliableChannelBinder`1.ChannelSynchronizer.TryGetChannel(Boolean canGetChannel, Boolean canCauseFault, TimeSpan timeout, MaskingMode maskingMode, TChannel& channel)
       at System.ServiceModel.Channels.ReliableChannelBinder`1.Send(Message message, TimeSpan timeout, MaskingMode maskingMode)
       at System.ServiceModel.Channels.SendReceiveReliableRequestor.OnRequest(Message request, TimeSpan timeout, Boolean last)
       at System.ServiceModel.Channels.ReliableRequestor.Request(TimeSpan timeout)
       at System.ServiceModel.Channels.ClientReliableSession.Open(TimeSpan timeout)
       at System.ServiceModel.Channels.ClientReliableDuplexSessionChannel.OnOpen(TimeSpan timeout)
       at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)
       at System.ServiceModel.Channels.ServiceChannel.OnOpen(TimeSpan timeout)
       at System.ServiceModel.Channels.CommunicationObject.Open(TimeSpan timeout)

    Exception rethrown at [1]:
       at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
       at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
       at System.ServiceModel.ICommunicationObject.Open()
       at ....InitializeClient() in D:\...\RcaClient.cs:line 298

     

     

    Again, this process works as expected when I use command line tools and load the client certificate from a .pfx file instead.  I have done a visual comparison in the debugger of the various fields that are null/not null between a X509Certificate2 loaded from the .pfx and from one loaded via the BouncyCastle library.  They are essentially the same.  Obviously some numerical values differ, but all the same values are present.  Any ideas why this exception is being thrown??

    Wednesday, June 1, 2011 2:12 PM

Answers

  • I started to go down that path (it was actually my preference), but then became aware that makecert/pvk2pfx are not redistributable.  That's why we went looking for a programmatic generation.

    I've kept poking at this and found something that does seem to fix the problem, I'm just not sure it is really what I am supposed to have to do.

    Bascially, the part that is causing problems is the PrivateKey I am assigning into the X509Certificate2.  That is currently being done by the line:

    dotnetCertificate2.PrivateKey = DotNetUtilities.ToRSA((RsaPrivateCrtKeyParameters)keyPair.Private);

    DotNetUtilities is part of the BoucyCastle library.  That line basically just takes their version of a private key and returns it as a RsaCryptoServiceProvider.  If, instead of using the returned provider, I create my own and copy the 'guts' into it:

          RSACryptoServiceProvider tempRcsp = (RSACryptoServiceProvider)DotNetUtilities.ToRSA((RsaPrivateCrtKeyParameters)keyPair.Private);
          RSACryptoServiceProvider rcsp = new RSACryptoServiceProvider(new CspParameters(1, "Microsoft Strong Cryptographic Provider", new Guid().ToString(), new CryptoKeySecurity(), null));
          rcsp.ImportCspBlob(tempRcsp.ExportCspBlob(true));
          dotnetCertificate2.PrivateKey = rcsp;
    

    That then works fine.

    The values I provided to the CspParameters constructer were more or less created based on what I saw in the debugger for the good certificate loaded from the file.  I have no idea if those values are truly right or if it is working by accident.

    Do you know what actually causes the "Key does not exist." exception?  I mean, there IS a private key object, so it does exist.

    • Marked as answer by Yi-Lun Luo Tuesday, June 7, 2011 9:24 AM
    Thursday, June 2, 2011 2:41 PM

All replies

  • OK, in comparing the two certificates further.  This time I was looking more at the values of things rather than null vs not null (which is what I THOUGHT I was looking for based on "Key does not exist."

    Things of note:

    certificate.PrivateKey:
    - _randomKeyContainer: False on the working; true on the not working
    - PersistKeyInCsp: True on the working; false on the not working

    certificate.PrivateKey._parameters (which is a CspParameters object):
    - Flags - UseExistingKey on the working certificate; CreateEphemeralKey on the not working
    - Provider Name - "Microsoft Strong Cryptographic Provider" on the working; "Microsoft Enhanced RSA and AES Cryptographic Provider" on the not working
    - ProviderType - 1 on the working; 24 on the not working

    certificate.PrivateKey.CspKeyContainerInfo:
    - Accessible - True on the working; false on the not working
    - Exportable - false on the working; throws "Key does not exist." on not working
    - KeyContainerName - has a GUID on the working; null on not working

     

    There are a few other differences, but these seemed to potentially be the most relevant - although I still don't understand "why" anything is the way it is, or what to do about it for that matter.

    Wednesday, June 1, 2011 3:10 PM
  • Hello, I haven't used BouncyCastle. But if you use Process.Start to start the makecert process programmatically, just like you did when you manually use the command line, does it work fine?
    Lante, shanaolanxing This posting is provided "AS IS" with no warranties, and confers no rights.
    Windows Azure Technical Forum Support Team Blog
    Thursday, June 2, 2011 3:18 AM
  • I started to go down that path (it was actually my preference), but then became aware that makecert/pvk2pfx are not redistributable.  That's why we went looking for a programmatic generation.

    I've kept poking at this and found something that does seem to fix the problem, I'm just not sure it is really what I am supposed to have to do.

    Bascially, the part that is causing problems is the PrivateKey I am assigning into the X509Certificate2.  That is currently being done by the line:

    dotnetCertificate2.PrivateKey = DotNetUtilities.ToRSA((RsaPrivateCrtKeyParameters)keyPair.Private);

    DotNetUtilities is part of the BoucyCastle library.  That line basically just takes their version of a private key and returns it as a RsaCryptoServiceProvider.  If, instead of using the returned provider, I create my own and copy the 'guts' into it:

          RSACryptoServiceProvider tempRcsp = (RSACryptoServiceProvider)DotNetUtilities.ToRSA((RsaPrivateCrtKeyParameters)keyPair.Private);
          RSACryptoServiceProvider rcsp = new RSACryptoServiceProvider(new CspParameters(1, "Microsoft Strong Cryptographic Provider", new Guid().ToString(), new CryptoKeySecurity(), null));
          rcsp.ImportCspBlob(tempRcsp.ExportCspBlob(true));
          dotnetCertificate2.PrivateKey = rcsp;
    

    That then works fine.

    The values I provided to the CspParameters constructer were more or less created based on what I saw in the debugger for the good certificate loaded from the file.  I have no idea if those values are truly right or if it is working by accident.

    Do you know what actually causes the "Key does not exist." exception?  I mean, there IS a private key object, so it does exist.

    • Marked as answer by Yi-Lun Luo Tuesday, June 7, 2011 9:24 AM
    Thursday, June 2, 2011 2:41 PM