none
URGENT: An error occurred when decrypting an AS2 message RRS feed

  • Question

  • hello,

    ok, i asked this question before and i still didnt get answers. i am stuck big time with this issue since weeks with a client. I have done all the possible solutions and nothing promising. I have recreated and re imported the certificate to the personal store several times and same problem...

    My client is trying to send us a simple file containing one word "test" , over AS2 protocol (http). and i am receiving the file and sending it to a location on my desk. i receive the file but with size of 0KB and the above error occurs. my client is singing using their certificate and encrypting the message using our public key. i have all certificates install in the right places.

    my client uses openAS2 software to encrypt the message using DES3 algorithm.

    Will you please help me what the possible causes of this problem??? as i am out of ideas now...

    I used certificate services to create the certificate. I can see the key usage for the private key does not include data encryption, but when i imported the pfx certificate using (certWizard utility) the key was imported successfully to the correct personal store ( the account running the iso host instance). and the key usage includes signing and encryption.... but the intended purpose is only for server authentication can this be the problem??

    Sorry for the long text but i am really stuck with this error and feel hopeless to find a solution..

    thanks.


    Regards, Mazin - MCTS BizTalk Server 2006


    Friday, August 3, 2012 9:58 AM

Answers

All replies

  • Can you try to set up a passthru and decrypt the messages outside of BizTalk, do you have the same issue? (just set a sendport with a filter towards the receive port).

    Can you also check and see if there are any errors in the event log?

    Best regards

    Tord Glad Nordahl
    Bouvet ASA, Norway
    http://www.BizTalkAdmin.com |@tordeman

    Please indicate ”Mark as Answer” if this post has answered the question.

    Friday, August 3, 2012 11:02 AM
  • Hi Tord,

    Thanks for your reply,

    how can i decrypt a message outside biztalk attaching the certificate?

    can you lead me the way please...

    If it takes so long to develop then i dont prefer this ...

    Another question, is there a best practice way on how to generate a certificate for encryptoin and signing using windows certificate services... since the private key (which is in the trusted root ) does not have key usage for encryption .... can that be the problem??? but when i imported the private key (pfx) using the certWizard ustility, it added the key to the personal store with key usage encryption and signing .. but if this key is the private key then why does it show " there is a private key that corresponds this certificate"

    if this is the problem then how can i add the KEY USAGE for  encryption .. is there a way to ammend the current private key to do so??

    i am really lost with the unsatisfying documentation about this...

    Thanks,


    Regards, Mazin - MCTS BizTalk Server 2006



    Friday, August 3, 2012 11:14 AM
  • I'm no AS2 guru (at all), so it might not even be possible, maybe someone else have some tips on that.

    However what we can do is to try and locate where the file goes "blank/empty" and try and drill down the problem.

    In BizTalk when you have global tracking turned on it will store information about in and out events, these events will contain information about the file, we also have tracking on the instances. You can also turn on different tracking options on the receive port to locate more information.

    In the BizTalkDTADb you can check the following tables dta_MessageInOutEvents you can use the following query:

    SELECT [dtTimestamp]
          ,[nStatus]
          ,[nMessageSize]
          ,[strUrl]
          ,[dtInsertionTimeStamp]
      FROM [BizTalkDTADb].[dbo].[dta_MessageInOutEvents]

    Also try and disable your send port and see if the filesize changes.

    Do you have any orchestration in this, or is it just a "simple" filetransfer?

    Best regards

    Tord Glad Nordahl
    Bouvet ASA, Norway
    http://www.BizTalkAdmin.com |@tordeman

    Please indicate ”Mark as Answer” if this post has answered the question.

    Friday, August 3, 2012 11:27 AM
  • Mazin just saw this in your text , seems incorrect to me.

    since the private key (which is in the trusted root )

    The private key should be in your personal store ( the personal store of app pool account). 

    Lets answer few basic question

    Is the public key for your partner in your Local computer\Other People store ?

    Is the private key inf the personal store of the account run by the App Pool?

    Is the client doing both signing and encryption? If yes then find the order and you are doing that in the reverse order.

    I would also suggest use a pass through pipeline first and get he message locally and then you can test with any configuration you want wihtout troubleing the client.

    I dont know know how to do it otherwise  but I would use pipeline.exe , (in this case you personal store).

    Friday, August 3, 2012 12:55 PM
  • Hi Sajid,

    So is it ok that the private key in the personal store shows "there is a private key corresponding this certificate..."??

    answering your questions:

    1- yes public key of partner in other ppl store

    2- yes the private key is in the personal store

    i found your next comment interesting... how can i know the order?? and what should i do if the order is in reverse? and what is the default order in BizTalk?

    thanks,


    Regards, Mazin - MCTS BizTalk Server 2006





    Friday, August 3, 2012 1:02 PM
  • Well on second thought, may that question doesnt seem relevant as both are using AS2.

    In biztalk you first verify the signature and then decrypt it. So your partner should virst be encrypting it and then signing. I beleive he is using some AS2 product so this must be taken care of but you can still verify.

    Also, Please paste the error message here, that might help as well.
    Friday, August 3, 2012 1:37 PM
  • Hi ,

    below is the error

    The AS2 Decoder encountered an exception during processing. Details of the message and exception are as follows: AS2-From:"PARTNER" AS2-To:"HOME" MessageID:"<OPENAS2-xxxxxxxxxxx>" MessageType: "unknown" Exception:"An error occurred when decrypting an AS2 message."


    Regards, Mazin - MCTS BizTalk Server 2006


    Friday, August 3, 2012 1:55 PM
  • Are you using the same account for running the app pool as the biztalk service account?
    • Marked as answer by Mazin Alassaf Monday, August 6, 2012 2:32 PM
    Friday, August 3, 2012 2:04 PM
  • I was not no.. I was using 2 different accounts, but i installed the the private key in both personal stores..

    .do you think that might be a problem??


    Regards, Mazin - MCTS BizTalk Server 2006

    Friday, August 3, 2012 2:14 PM
  • that is also fine but in that case, some times there is an issue of  user profile not loading.

    Please check out

    http://msdn.microsoft.com/en-us/library/gg634590.aspx

    • Marked as answer by Mazin Alassaf Monday, August 6, 2012 2:32 PM
    Friday, August 3, 2012 2:28 PM
  • Hi Sajid,

    I have already checked all these points with no success


    Regards, Mazin - MCTS BizTalk Server 2006

    Friday, August 3, 2012 3:06 PM
  • This is the inbound header information, not sure if it can give you more information

    Cache-Control: no-cache
    Connection: close, TE
    Date: Fri, 03 Aug 2012 15:47:42 +0100
    Pragma: no-cache
    Content-Length: 1560
    Content-Type: application/pkcs7-mime; name="smime.p7m"; smime-type=enveloped-data
    Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
    From: xxxxxxxxxx
    Host: xxxxxxxxxxxxxx
    User-Agent: OpenAS2 AS2Sender
    Message-ID: xxxxxxxxxxxxxxxxxxxxxxx
    Mime-Version: 1.0
    AS2-Version: 1.1
    Recipient-Address: xxxxxxxxxxxxxxxxxx
    AS2-To:xxxxxxxxx
    AS2-From: xxxxxxxxxxxxxxx
    Subject:  TEST
    Disposition-Notification-To: xxxxxxxxxxxxx
    Disposition-Notification-Options: signed-receipt-protocol=optional, pkcs7-signature; signed-receipt-micalg=optional, sha1


    Regards, Mazin - MCTS BizTalk Server 2006

    Friday, August 3, 2012 3:13 PM
  • in the suspended message do you see party resolved correctly?
    Friday, August 3, 2012 3:30 PM
  • Hi,

    I am getting a new error now :

    MessageType: "unknown" Exception:"An error occurred when validating an AS2 message. Make sure the certificates used have not timed out or been revoked."


    Regards, Mazin - MCTS BizTalk Server 2006

    Friday, August 3, 2012 3:50 PM
  • Hi,

    Two things to check

    1.  What's the serial number on the partner's cert?  If the serial number is 00 then Biztalk will not consider it valid.  Here's a link to a similar question

    http://social.msdn.microsoft.com/Forums/is/biztalkediandas2/thread/d4a8d620-b951-4345-8b3b-6800f740568a

    2.  Check the cert install.  Here's a quick guide I like 

    http://msdn.microsoft.com/en-us/library/gg634534.aspx

    Good Luck,

    Mike

    • Marked as answer by Mazin Alassaf Monday, August 6, 2012 2:31 PM
    Friday, August 3, 2012 6:02 PM
  • Hi Mike,

    You are right! the serial number is 00 .

    BUT! i am sending to this partner with no problems... the only problem is with receiving...

    what do you think?? does the serial number problem affect receiving only??

    Thanks Mike


    Regards, Mazin - MCTS BizTalk Server 2006

    Monday, August 6, 2012 9:03 AM
  • Hi,

    Did you had a look at the link,

    http://msdn.microsoft.com/en-us/library/bb898960(v=bts.20).aspx


    Thanks With Regards,
    Shailesh Kawade
    MCTS BizTalk Server
    Please Mark This As Answer If This Helps You.
    http://shaileshbiztalk.blogspot.com/

    Monday, August 6, 2012 9:19 AM
  • Hi Shailesh,

    Yes i looked at that many times, the problem is that the points are very general.. does not specify steps.


    Regards, Mazin - MCTS BizTalk Server 2006

    Monday, August 6, 2012 9:21 AM
  • Hi,

    I had the same problem receiving AS2 messages signed and/or encrypted with a cert with a serial number of 00.  Biztalk considers the cert invalid.

     I had to use the free AS2 connector http://www.rssbus.com/solutions/as2/ for the trading partner. You could also create your own AS2 process but I found that doing that was complicated.

    Thanks,

    mike

    • Marked as answer by Mazin Alassaf Monday, August 6, 2012 2:36 PM
    Monday, August 6, 2012 2:12 PM
  • Is this a bug in BizTalk server 2010 Mike?


    We dont want to use another software... since we just dumped OpenAS2 software for BizTalk


    Regards, Mazin - MCTS BizTalk Server 2006

    Monday, August 6, 2012 2:15 PM
  • I opened a  support incident with Microsoft and I was told the cert was invalid and there was not work around and the issue was not going to be fixed.  Other Microsoft products like Exchange, Outlook, Internet Explorer, and the certificate manager do not consider the cert invalid, but the Biztalk cert implementation is more strict.

    You could try to get your trading partner to generate a new cert with a serial number other than 00.

    Thanks,

    Mike

    • Marked as answer by Mazin Alassaf Monday, August 6, 2012 2:31 PM
    Monday, August 6, 2012 2:23 PM
  • Thanks Mike,

    I will inform my partner with these results. I will update this thread once i try with their new certificate (in case they agreed to generate a new one)

    Thanks for your help


    Regards, Mazin - MCTS BizTalk Server 2006

    Monday, August 6, 2012 2:36 PM