Running Database Unit Tests on a Solution Containing Multiple Test Projects

Question

I have a solution containing multiple database projects, as our application uses five different databases.  I Have created a Test Project for each Database Project in the solution.  Since the unit tests need to run as part of our CI process, I have created <MachineName>.dbunittest.config files to override the app.config files in each Test Project.  I am able to run the unit tests on our Build Server.  However, some of the tests fail because they cannot find the table the test is querying.  When I checked the Build Server, I realized that the <MachineName>.dbunittest.config file for one Test Project is overwriting the <MachineName>.dbunittest.config file of another Test Project.  Apparently, Microsoft does not expect anyone to use more than one Test Project per solution.

I am trying to workaround this problem, so I created a folder inside each Test Project and placed the <MachineName>.dbunittest.config files inside the folder.  I gave each folder a unique name so they would not be overwritten during deployment of the tests.  I opened LocalTestRun.testrunconfig and added the folders and the files in the "Deployment" section.  But when the build ran, it did not create the folders in the "Test Results" folder on the Build Server.  It still overwrote each <MachineName>.dbunittest.config file just like it did before.

I did find the folders I created in the "Binaries" folder on the Build Server, but that doesn't do me any good. 

How do I get the Build Server to create these folders in the "Test Results" folder where they need to be?

Answer 1

Hello Ken,

As far as I know, your solution can contain multiple test projects. Each test project contains one app.config file (that is, one set of configuration settings). As a result, your solution can contain different sets of database unit tests (one set for each test project) that are configured to run differently.

You can use the Database Test Configuration to specify the database connection strings and the schemas of one database project. Please refer to this article for more information:

http://msdn.microsoft.com/en-us/library/aa833430.aspx

I hope I can help you with your question.

Thanks,

---

Vicky Song [MSFT]
MSDN Community Support | Feedback to us
Get or Request Code Sample from Microsoft
Please remember to mark the replies as answers if they help and unmark them if they provide no help.

Answer 2

I'm not sure this applies in my case.  We're not running the tests through Visual Studio, we're running them as part of an automated build process.  We don't want to change the app.config because we need to use it for running test on our local machines.  The research I have done online says that you need to override the app.config and create the machinename.dbunittest.config files I mentioned. 

See this MSDN article for details: http://msdn.microsoft.com/en-us/library/aa833210.aspx

Since I have four test projects, I have four machinename.dbunittest.config files in my solution, all of which have the exact same name.  The article doesn't say if having multiple machinename.dbunittest.config files is allowed.  If could get me more information about this, it would be appreciated.

Thanks,

Ken

Answer 3

I figured it out.

The first Test Project I created was created in Visual Studio 2008.  When it was created, Visual Studio created a file named "LocalTestRun.testrunconfig".  The UI for this file looks identical to the UI for Local.testsettings.  I deleted LocalTestRun.testrunconfig and created a Local.testsettings file.  I configured this file the same way I configured LocalTestRun.testrunconfig.  When I deployed the solution with Local.testsettings, it was able to find the config files and the tests all passed.  Apparently there is a difference between LocalTestRun.testrunconfig and Local.testsettings.

I added comments to the article I referenced so that others will see that they need to remove their LocalTestRun.testrunconfig files.

 

Answer 4

I don't know how you were able to fix this.  I have the same problem and this solution was straight Visual Studio 2010 from the beginning.

In my two projects I have different connection strings.  One contains acceptance tests and the other integration tests.  To run these tests we use a different connection string because the data generation plans are different (for integration tests we use junk data in various random states, for integration tests we use blank slate real data).

When running on the build server the [machinename].dbunittest.config file always contains the configuration of the integration test project since that is the last one that gets deployed (alphabetically, I'm betting) in the "Deployment" settings of the Local.testsettings file.

Did I configure the deployment settings incorrectly?

Note I haven't seen anything called LocalTestRun.testrunconfig.

Also, I wanted to point out this:

http://connect.microsoft.com/VisualStudio/feedback/details/596793/datadude-limitation-of-dbunittest-config-files-for-db-unit-testing-need-more-than-one-per-server#tabs

Seems Microsoft knows of this issue but has it closed as "Won't Fix".

Answer 5

When I follow the steps outlined by Ken Powers in the link I get the exact same behavior as without the folders.  It seems that it doesn't deploy the folder, just copies the items in the folder to the deployment directory.  If I go to the deployment path (something like \\[BUILDSERVER]\c$\Builds\[BuildAgent]\[Project]\[BuildDefinition]\TestResults\BUILD 2011-06-16 11_39_04_Mixed Platforms_Debug\Out) I don't see the directories I added as deployment directories.  

Instead, I just see a single machinename.dbunittest.config file that is the latest one deployed out of the directories.

 

I tried to extend the workaround to put a folder within the folder.  So in my 2 database projects I'd have a path like the following: "DeploymentResources\<DatabaseName>TestConfig\machinename.dbunittest.config".  This DID deploy the directories so in my deployment directory I would see two folders, one for each database, with the appropriate configuration files inside.

However, when I run the tests I get errors saying it couldn't find the override configuration.  After the fact I realize this makes sense - how would the build server know to go into the correct directory to find the configuration?

I'm confused by how the workaround is even supposed to work.  HELP!!!!!!!!!!!

Answer 6

Ken, if you see these replies, maybe if you tell me what I should expect to see in the deployment folder I can figure out what I have wrong.  

Answer 7

Hi, sorry I couldn't respond sooner.  You caught me on a very busy day.

A couple things to check:

• Open your Test Project and check the properties of each .dbunittest.config file.  Each one should be set to "Copy Always" so they get copied to your Build Server
• Open the Local.testsettings file and click the Deployment tab
• Make sure the box "Enable Deployment" is checked

This next step may not make sense, but it worked for me:

DO NOT ADD ANY FILES OR FOLDERS to the Local.testsettings file

You may be having the same issue I had.  When I first created the Local.testsettings file, I used "Add File" and "Add Directory" buttons and entered the files and folders for all my dbunittest.config files.  Yet during deployment, the test engine kept reporting it could not find a configuration file.  As a troubleshooting step, I removed the files and folders from Local.testsettings and suddenly my tests were running.  I have not found any explanation for why it works like this, but it's been working for several months since I did this.

Now on your Build Server, you should have the Config folders you added to your Test Project under "Binaries\Release" along with the rest of your project files.

Hope this helps.

Ken

Answer 8

Ken,

I have been trying to follow this and have tried many combinations and nothing seems to work.

I have 2 folders with different names in 2 different test projects.

In those folders I have the machine specific config files that overrides the appconfig.

I have both of the machine specific config files set to Copy to Output Directory to Copy always

I see both of the folders with the respective files in the binaries\ folder I do not have a release folder?

In the local test settings I  have tried including the files and folders, just the files, just the folders, and nothing.. but nothing seems to work.

I was thiking about making 2 different testsettingsfiles ?