Running Test Plans with Multiple Data Sheets

Question

Hello.  Have been searching the forums all morning to see if anyone is in a similar situation to no avail.

I have a Project of roughly 250 CodedUI tests I need to run.

The test incorporate various part of our application: ordering, receiving, inventory, etc.  Our application (for reasons too complex to address here) runs against 12 different instances of the database, split by geographical region.  Each region has a two letter code.  The application itself is launched via Click Once by navigating to a web page and selecting a region, after which a Click Once link is provided which launches the application pointed at the appropriate database.

In most cases, regions do NOT share vendors and items, etc.  For example, ordering items 1234, 2345 and 3456 from vendor ABC will only work in ONE region.  Assuming those items are available in another region, they might come from a different vendor entirely, and have different IDs.  This necessitates using multiple test data spreadsheets, which we have.

The problem is:

I know I can set up test settings to place a particular data sheet in the drops directory, but as far as I can tell, the tests are still using the app.config to determine the path to that file.  I have several machines setup as a test farm and would like to be able to run the same set of tests against multiple regions at the same time.  Is there a way to specify some sort of parameter in the Test Settings which tells the tests which region to run against?  My test data sheets are named like:  TestData_XX_.xml where XX is the two letter region code.  

I want to be able to say:  

-Run these 200 tests against region AA

-Also run these 200 tests against region AB at the same time.

How can I accomplish this?

TIA.

Answer 1

Hi TIA,

I am trying to involve someone familiar with this topic to further look at this issue. There might be some time delay. Appreciate your patience.

Thank you for your understanding and support.

Best regards,

---

Lily Wu [MSFT]
MSDN Community Support | Feedback to us

Answer 2

Have you ever considered writing the region information to the test data, and consilidate data sheets from all regions?  You can also setup test lab environment. Setup test controller and a number of test agents, each agent run part of tests. I don't mean run test of a region on a test agent though.

thanks.

---

Forrest Guo | MSDN Community Support | Feedback to manager

Answer 3

Have you ever considered writing the region information to the test data, and consilidate data sheets from all regions?  You can also setup test lab environment. Setup test controller and a number of test agents, each agent run part of tests. I don't mean run test of a region on a test agent though.

thanks.

---

Forrest Guo | MSDN Community Support | Feedback to manager

I did think of this, but I still need a way to tell MTM which region I am running on.  It becomes a catch-22.  If I consolidate all the test data into one spreadsheet with multiple tabs, then I can set app.config to load that spreadsheet, but I still need a way to tell it which tab to pull data from.

If I keep the data in separate spreadsheets, then I have to tell app.config to load a particular sheet, which means I have to keep changing app.config to run against multiple regions.

Answer 4

To be clear, I mean consolidate the data sheets in one workbook, one sheet. I assume by "spreadsheet with multiple tabs" you mean multiple sheets.

I will try to get other people help this question, please consder above suggestion and report back.

thanks.

---

Forrest Guo | MSDN Community Support | Feedback to manager

Answer 5

Maybe you can group and run the Coded UI tests using Test Categories

http://msdn.microsoft.com/en-us/library/dd286683.aspx

I have not tested that but maybe a custom data collector that can get the TCM Information from the testcasestarteventargs and  pull the required data  which can then be consumed by the test method.
http://msdn.microsoft.com/en-us/library/microsoft.visualstudio.testtools.execution.testcasestarteventargs(v=vs.110).aspx
http://msdn.microsoft.com/en-us/library/microsoft.visualstudio.testtools.common.tcminformation(v=vs.110).aspx

---

Trevor Hancock (Microsoft)
Please remember to "Mark As Answer" the replies that help.