UnauthorizedAccessException when using MSDTC

Question

Hello everyone,

   We have a 3-tier application, the client, the server and the databaser server. It was developed using C# 2.0 and .Net 2.0, the database server is MS-SQL 2000 and we have some MSMQ in there. All 3 are located on different machines. The client is Windows XP SP-2, the application server is Windows Server 2003 SP1 and the daatabase server is also Windows server 2003 SP1.

 

    Most of the time when the client performs an operation to the application server the database gets updated and something is written to a queue. The writing to the database and queues is performed in the context of a transaction, we use the System.Transactions facilities and MSDTC. When the server performs the writing operation it 'impersonates' the client, although this should not have any impact since our database connection string contains a database user id and password.

 

   This works great in almost all our test sites... but lately we have run some tests using VM's for the client and the server application. Under some specific configuration we have run into a situation where we would get an exception:

---

      Inner Exception: UnauthorizedAccessException
      Properties:
      Message = 'Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))'
            Data (Dictionary with 0 entrie(s))
            TargetSite = 'Void ConnectToProxy(System.String, System.Guid, IntPtr, Boolean ByRef, UInt32 ByRef, System.Transactions.Oletx.CoTaskMemHandle ByRef, System.Transactions.Oletx.IResourceManagerShim ByRef)'
            HelpLink = null
            Source = 'System.Transactions'
            Stack Trace:
         at System.Transactions.Oletx.IDtcProxyShimFactory.ConnectToProxy(String nodeName, Guid resourceManagerIdentifier, IntPtr managedIdentifier, Boolean& nodeNameMatches, UInt32& whereaboutsSize, CoTaskMemHandle& whereaboutsBuffer, IResourceManagerShim& resourceManagerShim)
         at System.Transactions.Oletx.DtcTransactionManager.Initialize()
         at System.Transactions.Oletx.DtcTransactionManager.get_ProxyShimFactory()
         at System.Transactions.Oletx.OletxTransactionManager.CreateTransaction(TransactionOptions properties)
         at System.Transactions.TransactionStatePromoted.EnterState(InternalTransaction tx)
         at System.Transactions.EnlistableStates.Promote(InternalTransaction tx)
         at System.Transactions.Transaction.Promote()
         at System.Transactions.TransactionInterop.ConvertToOletxTransaction(Transaction transaction)
         at System.Transactions.TransactionInterop.GetExportCookie(Transaction transaction, Byte[] whereabouts)
         at System.Data.SqlClient.SqlInternalConnection.EnlistNonNull(Transaction tx)
         at System.Data.SqlClient.SqlInternalConnection.Enlist(Transaction tx)
         at System.Data.SqlClient.SqlInternalConnectionTds.Activate(Transaction transaction)
         at System.Data.ProviderBase.DbConnectionInternal.ActivateConnection(Transaction transaction)
         at System.Data.ProviderBase.DbConnectionPool.GetConnection(DbConnection owningObject)
         at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
         at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
         at System.Data.SqlClient.SqlConnection.Open()

---

 

    We have tripple checked the MSDTC configuration, we have run DTCping, we cannot find the problem.

 

We have also activated the 'Audit on Failure' and we got these errors in the event viewer:

 

BLITZ is our domain controller
C8SERVER is our application server
dispatcher2 is the user logged on the client pc.

Event Type: Failure Audit
Event Source: Security
Event Category: Object Access
Event ID: 560
Date:  15/08/2006
Time:  1:41:28 PM
User:  BLITZ\dispatcher2
Computer: C8SERVER
Description:
Object Open:
  Object Server: SC Manager
  Object Type: SC_MANAGER OBJECT
  Object Name: ServicesActive
  Handle ID: -
  Operation ID: {0,19314977}
  Process ID: 452
  Image File Name: C:\WINDOWS\system32\services.exe
  Primary User Name: C8SERVER$
  Primary Domain: BLITZ
  Primary Logon ID: (0x0,0x3E7)
  Client User Name: dispatcher2
  Client Domain: BLITZ
  Client Logon ID: (0x0,0x126A67F)
  Accesses: READ_CONTROL
   Connect to service controller
   Enumerate services
   Query service database lock state

  Privileges: -
  Restricted Sid Count: 0
  Access Mask: 0x20015


For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

____________________________________________________________________________________________________________
Event Type: Failure Audit
Event Source: Security
Event Category: Object Access
Event ID: 560
Date:  15/08/2006
Time:  1:41:28 PM
User:  BLITZ\dispatcher2
Computer: C8SERVER
Description:
Object Open:
  Object Server: SC Manager
  Object Type: SC_MANAGER OBJECT
  Object Name: ServicesActive
  Handle ID: -
  Operation ID: {0,19314980}
  Process ID: 452
  Image File Name: C:\WINDOWS\system32\services.exe
  Primary User Name: C8SERVER$
  Primary Domain: BLITZ
  Primary Logon ID: (0x0,0x3E7)
  Client User Name: dispatcher2
  Client Domain: BLITZ
  Client Logon ID: (0x0,0x126A67F)
  Accesses: READ_CONTROL
   Connect to service controller
   Enumerate services
   Query service database lock state

  Privileges: -
  Restricted Sid Count: 0
  Access Mask: 0x20015


For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

____________________________________________________________________________________________________________

Event Type: Failure Audit
Event Source: Security
Event Category: Object Access
Event ID: 560
Date:  15/08/2006
Time:  1:41:28 PM
User:  BLITZ\dispatcher2
Computer: C8SERVER
Description:
Object Open:
  Object Server: SC Manager
  Object Type: SERVICE OBJECT
  Object Name: MSDTC
  Handle ID: -
  Operation ID: {0,19315216}
  Process ID: 452
  Image File Name: C:\WINDOWS\system32\services.exe
  Primary User Name: C8SERVER$
  Primary Domain: BLITZ
  Primary Logon ID: (0x0,0x3E7)
  Client User Name: dispatcher2
  Client Domain: BLITZ
  Client Logon ID: (0x0,0x126A67F)
  Accesses: Query status of service

  Privileges: -
  Restricted Sid Count: 0
  Access Mask: 0x4


For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

____________________________________________________________________________________________________________

Event Type: Failure Audit
Event Source: Security
Event Category: Object Access
Event ID: 560
Date:  15/08/2006
Time:  1:41:28 PM
User:  BLITZ\dispatcher2
Computer: C8SERVER
Description:
Object Open:
  Object Server: SC Manager
  Object Type: SERVICE OBJECT
  Object Name: MSDTC
  Handle ID: -
  Operation ID: {0,19315253}
  Process ID: 452
  Image File Name: C:\WINDOWS\system32\services.exe
  Primary User Name: C8SERVER$
  Primary Domain: BLITZ
  Primary Logon ID: (0x0,0x3E7)
  Client User Name: dispatcher2
  Client Domain: BLITZ
  Client Logon ID: (0x0,0x126A67F)
  Accesses: Query status of service

  Privileges: -
  Restricted Sid Count: 0
  Access Mask: 0x4


For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

____________________________________________________________________________________________________________

Event Type: Failure Audit
Event Source: Security
Event Category: Object Access
Event ID: 560
Date:  15/08/2006
Time:  1:41:28 PM
User:  BLITZ\dispatcher2
Computer: C8SERVER
Description:
Object Open:
  Object Server: SC Manager
  Object Type: SERVICE OBJECT
  Object Name: MSDTC
  Handle ID: -
  Operation ID: {0,19315260}
  Process ID: 452
  Image File Name: C:\WINDOWS\system32\services.exe
  Primary User Name: C8SERVER$
  Primary Domain: BLITZ
  Primary Logon ID: (0x0,0x3E7)
  Client User Name: dispatcher2
  Client Domain: BLITZ
  Client Logon ID: (0x0,0x126A67F)
  Accesses: Query status of service

  Privileges: -
  Restricted Sid Count: 0
  Access Mask: 0x4


For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

____________________________________________________________________________________________________________

Event Type: Failure Audit
Event Source: Security
Event Category: Object Access
Event ID: 560
Date:  15/08/2006
Time:  1:41:28 PM
User:  BLITZ\dispatcher2
Computer: C8SERVER
Description:
Object Open:
  Object Server: SC Manager
  Object Type: SC_MANAGER OBJECT
  Object Name: ServicesActive
  Handle ID: -
  Operation ID: {0,19315269}
  Process ID: 452
  Image File Name: C:\WINDOWS\system32\services.exe
  Primary User Name: C8SERVER$
  Primary Domain: BLITZ
  Primary Logon ID: (0x0,0x3E7)
  Client User Name: dispatcher2
  Client Domain: BLITZ
  Client Logon ID: (0x0,0x126A67F)
  Accesses: Connect to service controller
   Query service database lock state

  Privileges: -
  Restricted Sid Count: 0
  Access Mask: 0x11


For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

____________________________________________________________________________________________________________

Event Type: Failure Audit
Event Source: Security
Event Category: Object Access
Event ID: 560
Date:  15/08/2006
Time:  1:41:28 PM
User:  BLITZ\dispatcher2
Computer: C8SERVER
Description:
Object Open:
  Object Server: SC Manager
  Object Type: SERVICE OBJECT
  Object Name: MSDTC
  Handle ID: -
  Operation ID: {0,19315273}
  Process ID: 452
  Image File Name: C:\WINDOWS\system32\services.exe
  Primary User Name: C8SERVER$
  Primary Domain: BLITZ
  Primary Logon ID: (0x0,0x3E7)
  Client User Name: dispatcher2
  Client Domain: BLITZ
  Client Logon ID: (0x0,0x126A67F)
  Accesses: Query service configuration information

  Privileges: -
  Restricted Sid Count: 0
  Access Mask: 0x1


For more information, see Help and Support Center at
http://go.microsoft.com/fwlink/events.asp.

____________________________________________________________________________________________________________

 

At the time we thought that this was a Virtual server specific problem, but we see one of our customer with the same problem on and off. The customer will exprience the same problem for a short period of time on a daily basis. We are currently trying to figure what goes on at this specific location.

 

Can anybody shed some ligth or give us a clue as what migth cause this problem?

 

Best Regards

 

-Martin

 

Answer 1

You might be hitting a issue fixed in the latest COM+ QFE: http://support.microsoft.com/kb/908473/

Can you try installing the hotfix mentioned at the end of the article to see if the issue goes away?

HTH

Answer 2

Thank you Florin,

    I'll try to get my hands on the hotfix and I'll let you know.

 

-Martin

Answer 3

Hi Florin,

    I just installed the hotfix in our lab and it did not resolve the issue. One thing I forgot to mention is that the server in our is running as a virtual machine, while our customeris running a 'real' windows 2003 server.

 

    I'll continue investigating, if you have any other ideas they are more than welcome.

 

-Regards

Martin

Answer 4

Hi Martin,

Sorry to hear it doesn't fix your problem.

Can you give more details on what is the special config you are seeing this issue: "This works great in almost all our test sites... but lately we have run some tests using VM's for the client and the server application. Under some specific configuration we have run into a situation where we would get an exception"?

Is it an intermittent issue or in that particular config, it reproduces all the time?

Thanks!

Answer 5

Ok, here it goes....

 

In our lab we have always tested our server application on real machines running Windows Server 2003 SP1, we always use the same database server. We have never experienced that problem in these kind of setups.

About 3 weeks ago we started testing using virtual machines. It tooks us a while to realize that the version of Windows Server 2003 that we were using was not SP1... this is when the problem started to appear. We have upgraded on of our virtual servers to SP1 and from that point on we have encountered that problem. Note that in our lab this problem is not intermitent. We have this problem every single time.

 

Now, last week we have deployed at a customer site. The customer is running the server application on a Windows 2003 Server R2 SP1(whatever R2 means). We are experiencing this situation every day between 10:30am  and noon....  I know this sounds very stupid but that is the fact. The rest of the day the application runs without a problem. To circumvent this problem we have changed our connection string and have set "Enlist=false".

 

So bottom line is that we were not too worried because we were seeing this behavior only when running on virtual machine, now we are more concerned since this occurs in a production environment and a real server.

I hope this clarifies the situation.

 

Thank you for your help

 

-Martin

Answer 6

The fact that your customer is seeing the issue between 10:30am and 12:00pm is very strange. Is there anything else happening in that period of time? Like backups/domain controllers rebooted etc?

 

As for the virtual machines issues, what SKU of Windows Server 2003 are you using on the virtual machine? Is it by chance the Web Server SKU?

 

Can you run the following command on the virtual machine experiencing the access denied issue and post back the output:

 

      "sc sdshow msdtc"

 

Thanks.

Answer 7

Hi Florin,

    I am not familiar with the term SKU, we are running "Windows Server 2003 Standard Edition Service Pack 1".

    Here is the output you requested.  

D:(A;;CCLCSWRPLOCRRC;;;S-1-2-0)(A;;CCLCSWRPWPDTLOCRRC;;;SY)(A;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;BA)(A;;CCLCSWLOCRRC;;;IU)(A;;CCLCSWLOCRRC;;;SU)(A;;CR;;;AU)(A;;CCLCSWRPWPDTLOCRRC;;;PU)(A;;CCLCSWRPLORC;;;NS)S:(AU;FA;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;WD)

 

  Thank you for your help.

We are still trying to figure out what is going on between 10:30 and 12:00... the customer claims that not much special is going on, it needs be I will try to put a sniffer to see what is happening.

 

-Martin

Answer 8

Hi Martin,

 

Somebody/Something messed up the security access for Authenticated Users. You have:

            (A;;CR;;;AU)

And it should be something similar to:

            (A;;CCLCSWRPLOCRRC;;;AU)

 

Here are a few pointers with descriptions:

SC command: http://technet2.microsoft.com/WindowsServer/en/library/1e71d513-bc67-4928-889f-9654f5afbfab1033.mspx?mfr=true

ACE Strings: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/secauthz/security/ace_strings.asp

 

The simplest way to do this is to run the following command in one line (no spaces in the ACL list):

 

"sc sdset MSDTC D:(A;;CCLCSWRPLOCRRC;;;S-1-2-0)(A;;CCLCSWRPWPDTLOCRRC;;;SY)(A;;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;BA)(A;;CCLCSWLOCRRC;;;IU)(A;;CCLCSWLOCRRC;;;SU)(A;;CCLCSWRPLOCRRC;;;AU)(A;;CCLCSWRPWPDTLOCRRC;;;PU)(A;;CCLCSWRPLORC;;;NS)S:(AU;FA;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;WD)"

 

NOTE: For other people looking at the sc sdset command - please do not use it as is, since this ACL list is specific to Martin's machine. What I did was to replace (A;;CR;;;AU) with (A;;CCLCSWRPLOCRRC;;;AU) in his original output from "sc sdshow MSDTC".

 

I hope this helps.

Answer 9

Thank you Florin,

   Changing the secutiry descriptior did the trick!!!  We will keep a close watch on the sd when we will create a new virtual machine, just in case.

 

   This does not resolve the issue at our custmoer site, but we will monitor this situation as weel.

 

Thank you!

Answer 10

Outstanding - I was about to open a support ticket when I came across this.  This was our exact problem as well and the proposed solution is working in our test environment.

 

Thanks!

 

Lane

Answer 11

This definitely helps.

 

We have been suffered from the same issue in a similar situation.

 

The fix works on our production server which has different sc sdshow MSDTC dump to our development server.

 

The suggested fix '(A;;CCLCSWRPLOCRRC;;;AU)' appears on our development server sc sdshow MSDTC dump so we replaced the '(A;;CR;;;AU)' ACE string on the production and get MSDTC back to work.

 

We suspect that the '(A;;CR;;;AU)' ACE is caused by 'Something' like hot fix application because all of our MSDTC GUI configuration research doesn't seem to make Authenticated User receiving such rights.

 

Kevin Chiang

Answer 12

Hi Florin,

 

I just found this post and got the same issue with our application. Two servers have different settings when I tried this: "sc sdshow msdtc".

 

DEV ENV:

 

D A;;CCLCSWRPLOCRRC;;;S-1-2-0)(A;;CCLCSWRPWPDTLOCRRC;;;SY)(A;;CCDCLCSWRPWPDTLOC

RSDRCWDWO;;;BA)(A;;CCLCSWLOCRRC;;;IU)(A;;CCLCSWLOCRRC;;;SU)(A;;CR;;;AU)( A;;CCLCS

WRPWPDTLOCRRC;;;PU)(A;;CCLCSWRPLORC;;;NS)S AU;FA;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;WD)

BUILD ENV:

D: (A;;CCLCSWRPLOCRRC;;;S-1-2-0)(A;;CCLCSWRPWPDTLOCRRC;;;SY)(A;;CCDCLCSWRPWPDTLOC

RSDRCWDWO;;;BA)(A;;CCLCSWLOCRRC;;;AU)(A;;CCLCSWRPWPDTLOCRRC;;;PU)(A;;CCLCSWRPRC;

;;WD) (A;;CCLCSWRPLORC;;;NS)S AU;FA;CCDCLCSWRPWPDTLOCRSDRCWDWO;;;WD)

There are 2 different strings. In the answer to this post I found that it has to be (A;;CCLCSWRPLOCRRC;;;AU) but our "working" server has  (A;;CCLCSWRPLOCRRC;;;WD) where WD is WRITE_DAC and AU - SYSTEM_AUDIT_ACE_TYPE.

I'm confused if I should perform setting to (A;;CCLCSWRPLOCRRC;;;AU) or to (A;;CCLCSWRPLOCRRC;;;WD) as on the "working" server?

Could you please advice?

 

Thanks

Answer 13

Florin!

 

Thank you so much for this post! I wanted to solve my problem as soon as possible and decided to just change it to the way it was on my dev machine for only that one string (A;;CR;;;AU) vs (A;;CCLCSWRPRC; ;;WD)

 

It worked! And now transaction is not being aborted anymore because of System.UnauthorizedAccessException: Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED))

 

This took me a while to research but now at least I will know where to look at when Transaction is being aborted. Also, I assume this could happen at anytime after the updates are downloaded and installed? It happened to us before - our orders could not be saved with the same exception being thrown.

 

Thanks

Answer 14

 

Hi Tatyana,

I am glad that this helped!

 

Have a nice day!

-Martin

Answer 15

Hi Tatyana,

We are not able to reproduce the problem on our side, i.e. we don't know what it changing the ACLs on your machines.
If you or anybody else hitting this issue can identify what is changing the ACLs, we will be happy to investigate further.

For instance if you can identify that the ACLs are being changed after installing a specific Windows update (like KBnnnnnn), that would be best. We have no evidence at this point that shows that Windows updates are causing this.

Thanks!

Answer 16

Hi Florin,

 

Yes I agree that saying that its because of the hot fix or updates is not right. But I asked our engeneers to notify me when the servers are going to be updated and anything gets installed so we can review the MSDTC variables before and after a restart. This is not the first time this hppened to us.

 

Thank you again for all the help and this great forum post!

 

Thanks

 

 

Answer 17

 

I've tried both resolution above - no luckL

 

here is my scmanager and msdtc output - (p.s. The issue I'm having is we have our help desk trying to edit smtp addresses, exchange rights have been granted, access toOU as well).  Once we removed the last W2K DC and now are all W2K3 DC SP2, they cannot modify SMTP addresses where they could prior to this.  

 

scmanager:

DA;;CCLCRPRC;;;AU)(A;;CCLCRPRC;;;IU)(A;;CCLCRPRC;;;SU)(A;;CCLCRPWPRC;;;SY)(A;;
KA;;;BA)SAU;FA;KA;;;WD)(AU;OIIOFA;GA;;;WD)

 

msdtc:

 DA;;CCLCSWRPLOCRRC;;;AU)(A;;CCLCSWRPWPDTLOCRRC;;;SY)(A;;CCDCLCSWRPWPDTLOCRSDRC
WDWO;;;BA)(A;;CCLCSWLOCRRC;;;IU)(A;;CCLCSWLOCRRC;;;SU)(A;;CCLCSWRPLOCRRC;;;AU)(A
;;CCLCSWRPWPDTLOCRRC;;;PU)(A;;CCLCSWRPLORC;;;NS)SAU;FA;CCDCLCSWRPWPDTLOCRSDRCW
DWO;;;WD)

 

One help desk resource can edit no problem - the other one cannot.  Both are members of the same groups, local admin on their own W2K Pro box and even the same with Windows XP, with E2K3 admin Sp2 installed and AD Support Tools. No individual permissions granted for either.

 

There is an event 560 logged on E2K3 server when the unsuccessful user tries to modify SMTP addresses.

I tried both resolutions in KB article for setting the SA permissions but no luck.  http://support.microsoft.com/?id=905809 

The user receives the exact error in the article....

 

 

Event Type: Failure Audit
Event Source: Security
Event Category: Object Access
Event ID: 560
Date:  10/9/2008
Time:  12:28:52 PM
User:  BIONET\ca1jgallant
Computer: Exchange

Description:
Object Open:
  Object Server: SC Manager
  Object Type: SERVICE OBJECT
  Object Name: MSExchangeSA
  Handle ID: -
  Operation ID: {0,623732}
  Process ID: 500
  Image File Name: C:\WINDOWS\system32\services.exe
  Primary User Name: Exchange$
  Primary Domain: BIONET
  Primary Logon ID: (0x0,0x3E7)
  Client User Name: ca1jgallant
  Client Domain: BIONET
  Client Logon ID: (0x0,0x98462)
  Accesses: Query status of service
   
  Privileges: -
  Restricted Sid Count: 0
  Access Mask: 0x4

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Thanks for any help:

 

Glenn

Answer 18

Glenn,

 

Have you tried to look at the different machine's  scmanager: and msdtc variables output? I remember I did the same on the staging server and compared to failing production, then modified the production according to staging. Worked for me. Also find out if there were recent updates or upgrades done to the machine. Most likely that's the case...

 

Thanks,

Tatyana

Answer 19

 

the outputs I listed are from the single Exchange server in our test lab.   I use the same workstaion for logging on with the two different ID's to test modifying smtp addresses.   Which machines do you want me to compare against?

 

BTW - if I grant the unsuccessful user local admin on the Exchange box, it works of course.  I just need elevated rights for the system attendant to have this work, somehow one user has the rights and the other don't.

 

Thanks

 

Glenn