locked
DataDriven Tests - Relative Path of the Data File Error - Unit test adpter cannot connect to the data source.. RRS feed

  • Question

  • Visual Srudio 2010 had a TestView and each test case had a proeprty window.

    You can use that to set the data source for the data-driven unit tests.

    That is missing in VS 2012. Is that feature dropped in 2012?

    I am having a hard time porting some data-driven unit tests from 2010 to 2012.

    It is due to the complex solution folder structure.

    Since I can't upload my source code that can replicate the issue, I am trying to explain it below.

    I have the solution set up as follows.

    The path of the CSV file related to the solution is :

    ABC Communication Services\ABC.ABCCommunicationServices.Test1\CT\CSVTestData.csv

    And the test case looks like this.

    [DeploymentItem("ABC Communication Services\\ABC.ABCCommunicationServices.Test1\\CT\\ABCTestData.csv"), TestMethod]
            [DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", "|DataDirectory|\\CSVTestData.csv", "CSVTestData#csv", DataAccessMethod.Sequential)]
            public void TestDrivenTestWithCSVWorking()
            {
                Arith target = new Arith();
                int valueA = Convert.ToInt32(TestContext.DataRow["arg1"]);
                int valueB = Convert.ToInt32(TestContext.DataRow["arg2"]);
                int expectedResult = Convert.ToInt32(TestContext.DataRow["expectedResult"]);
                int actualResult = target.AddIntegers(valueA, valueB);
                Assert.AreEqual(expectedResult, actualResult, "The result is wrong.");
                //Additional Message - Data Conditions Tested
                testContextInstance.WriteLine(TestContext.DataRow["dataCondition"].ToString() + Environment.NewLine);
            }
    

    The CSV file looks like this:

    arg1,arg2,expectedResult,dataCondition

    1,1,2,positive

    -2,-2,-4,negative

    0,-1,-1,zero and negative

    I am getting the following error while running the test.

    The unit test adapter cannot connect to the datasource to read the data.

    Can anybody please suggest me a way to resolve this issue?

    Thanks in advance!

    Wednesday, March 6, 2013 11:34 PM

Answers

  • Hello Sudheer,

    Since there is no wizard like in VS2010 used to set the datasource for test method in VS2012, we have to add the datasource attribute and DeploymentItem attribute for test method manually in VS2012. In this case we need consider the path of the deployed file to configure DeploymentItem attribute.

    I don’t think that it is an issue in VS2012. Configuring the appropriate DeploymentItem attribute based on the files located on different directories and “Copy to OutputDirectory” property can make tests successful on my side.

    If you want VS2012 to perform the same way with VS2010 and don't want to manually select the target test settings file, you could submit this feature request:

    http://visualstudio.uservoice.com/forums/121579-visual-studio

    The Visual Studio product team is listening to user voice there. You can send your ideas/suggestions there and people can vote.

    Thanks for your understanding.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    • Marked as answer by Sudheer Kumar Tuesday, March 19, 2013 11:45 PM
    Tuesday, March 19, 2013 1:50 AM

All replies

  • I have an update on this issue.

    I added a test settings file to the solution, (I don't know how to create a new one, added one from an another solution)

     and did the following setting on the deployment tab.

    I was under the impression that, if I mention : DeploymentItem() on the attribute, I don't have to do this.

    Is this the right way to resolve this issue?

    Any suggestions?

    Thursday, March 7, 2013 3:57 PM
  • Hello Sudheer,

    Thank you for your post.

    Yes. There is no wizard like in VS2010 used to set the datasource for test method in VS2012. We have to add the datasource attribute for test method manually in VS2012.

    This blog below gives sample Data Source strings. Copy them to your code and make the necessary customizations.

    http://blogs.msdn.com/b/mathew_aniyan/archive/2009/03/17/data-driving-coded-ui-tests.aspx
    In addition, you can try to check if you have set the
    CSV file property Copy to Output Directory to ‘Copy Always’. If not, please change it to check the result.

    You can refer to the following thread to know more about CSV file as datasource:

    http://social.msdn.microsoft.com/Forums/en-US/vsautotest/thread/40333bb4-53cd-4c9e-8c09-c0c552b28daf/

    If still no help, please try to use the whole path of the CSV file in Deployment attribute rather than relative path to check if it can help.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Friday, March 8, 2013 3:29 AM
  • Thanks Amanda for your response.!

    I tried the option : "Copy Always" and found that it works in some scenarios, but not always.

    Here are my findings.

    Scenario 1 - CSV File directly under Project. (Solution=>Project=>CSV File):

    1. Set "Copy Always" on the CSV file

    2. Run the test and it succeeds

    3. Check the "TestResults" folder for the current run. (Go to the solution folder->TestResults folder, each run will create a folder)

    You can see the CSV file and dll are being copied to the Out directory and you know why it works.

    I have the same test files as I mentioned before.

    Scenario 2 - CSV File under a folder in the project (Solution=>Project=>Folder=>CSV File):

    1. Set "Copy Always" on the CSV file residing under the folder.

    2. Run the test and it fails.

    3. Check the "TestResults" folder for the current run. (Go to the solution folder->TestResults folder, each run will create a folder)

    I found that the "Out" folder for the current run is empty, not even the dll is present there. Obviously it won't work.

    Scenario 3 - Project inside a Solution folder (Solution=>Solution Folder=>Project=>Folder=>CSV File):

    1. Set "Copy Always" on the CSV file residing under the folder.

    2. Run the test and it fails.

    3. Check the "TestResults" folder for the current run. (Go to the solution folder->TestResults folder, each run will create a folder)

    I found that the "Out" folder contains the compiled Dll, but not the CSV and hence it fails.

    I have the following conclusions based on this.

    1. The DeploymentItem attribute doesn't seem to be working in VS 2012 (I have Ultimate)

    2. The "Copy Always" will work for some scenarios (eg: Scenario 1) but not for all cases.

       I commented out the DeploymentItem attribute and still it continue working just with "Copy Always" setting, which means DeploymentItem has no effect.

    3. The work around I suggested works for all configurations. You don't need to do "Copy Always" along with this.

    I can send the sample source code it is not reproducible.

    Please correct me if I am wrong.


    Friday, March 8, 2013 9:47 PM
  • Hello Sudheer,

    Glad to receive your reply.

    The work around I suggested works for all configurations. You don't need to do "Copy Always" along with this.

    If the workaround can work on all your scenarios, I think that it is a way to resolve this issue.

    As for your scenario2 and scenario 3, I have tried them with a simple data driven unit test in VS2012 Ultimate and test still can work when setting "Copy Always’ on the CSV file’. You can try the string in Datasource below with your scenarios to check if they can help.

    Scenario 2: Solution=>Project=>Folder=>CSV File

    Datasource string:      

     [DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", "|DataDirectory|\\data1.csv", "data1#csv", DataAccessMethod.Sequential), DeploymentItem("NewFolder1\\data1.csv"), TestMethod]

    Scenario 3: Solution=>Solution Folder=>Project=>Folder=>CSV File

    Datasource string:      

     [DataSource("Microsoft.VisualStudio.TestTools.DataSource.CSV", "C:\\Users\\XXX \\Documents\\Visual Studio 2012\\Projects\\TestProject1\\UnitTestProject3\\NewFolder\\data2.csv", "data2#csv", DataAccessMethod.Sequential), DeploymentItem("Testproject\\NewSolutionFolder\\UnitTestProject3\\NewFolder\\data2.csv"), TestMethod]

    This path(C:\\Users\\XXX \\Documents\\Visual Studio 2012\\Projects\\TestProject1\\UnitTestProject3\\NewFolder\\data2.csv) is the full path of data2.csv. You can copy from the Full Path in Properties window of data2.csv file.

    I commented out the DeploymentItem attribute and still it continue working just with "Copy Always" setting, which means DeploymentItem has no effect.

    For scenario 2, if I commend out the DeploymentItem attribute, I have to change |DataDirectory|\\data1.csv to |DataDirectory|\\NewFolder\\data1.csv to make sure the datasource can be found.

    For scenario 3, since I define full path of the csv file in DataSource we can commend out the DeploymentItem and test still can work.

    Also I have tried to change |DataDirectory|\\NewFolder\\data1.csv in scenario2 to the full path of csv file and set ‘Copy Always’. I found that the test still can work when we recommend out the DeploymentItem.

    So if you want to recommend out the DeploymentItem attribute, you need to specify the full path of csv file in DataSource attribute and set ‘Copy Always’to make sure test can work.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Monday, March 11, 2013 2:38 AM
  • Thanks Amanda again for the details reply.

    I am sorry that I got delayed on my response.

    I still can't get you solution working for me. So what I did is to create sample in VS 2010 and replicate the same on VS 2012.

    VS 2010 Sample:

    http://code.msdn.microsoft.com/Data-Drive-Unit-Tests-420a05ea

    VS 2012 Sample:

    http://code.msdn.microsoft.com/Data-Drive-Unit-Tests-4b7df73d

    Why doesn't the 2012 sample works as it used to be in 2010.

    Also as mentioned in the sample, I don't agree using absolute paths in the DeploymentItem or in DataSource attributes as that will give you trouble when builing on another machine or build machine.

    Please let me know if you find my observations incorrect.

    ************NOTE**********************

    The usage of a test settings file is not an ideal solution.

    I had a huge solution which already had a test settings file that has lists out code coverage etc in it.

    If I use the same test settings file and when I try to debug a test case locally, it takes too long to reach my code. It seems to be loading all those dlls or doing something when I try to debug locally.

    Due to this, I had to create a new testsetting file and add the testdata files to it.

    Due tho this issue, I wish if it has worked the same way as it was in VS 2010.

    Thanks again for following up on this issue!

    Thursday, March 14, 2013 6:57 PM
  • Hello Sudheer,

    Glad to receive your reply.

    When ever you add a test project, VS 2012 doesn't add a test settings file. I guess 2012 enables deployment by default.

    As far as I know, in VS2012 we can deploy an item just through using DeploymentItemAttribute without a .testsettings file. For more information, please see: http://msdn.microsoft.com/en-us/library/vstudio/ms182475.aspx

     “Also please don't use full paths in DeploymentItem() or DataSource() because on the build machine the paths could be totally different.”

    Since this consideration, you don’t want to use the full paths in DeploymentItem. But based on my tries, I am afraid that the DeploymentItem will not work if you don’t use the full path for scenario 3. Generally we put the deployed files directly under Project file, the DeploymentItem attribute will work well. For your different scenarios, since the directory of the file is really different and much related to the solution directory and project directory, maybe we need some changes on DeploymentItem for different scenarios as I tried in my previous reply.

    Through your samples, it seems that since using DeploymentItemAttribute and setting "Copy to Output Directory: Copy Always" on the test data file does not works for all of your scenarios, so you set the .csv file property : Copy to Output Directory  to ‘Donot Copy’ and add a testsettings file to the solution for deploying files in order to make all scenarios can work. If this can work well with all your scenarios, I think that it is a feasible workaround.

    In addition, thank you for sharing your solutions & experience here. It will be very beneficial for other community members who have similar questions.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Friday, March 15, 2013 2:25 AM
  • Thanks Amanda for your inputs!

    The point I was trying to make with my sampels was that the DeploymnetItem( ) attribute works for all possible combinations in Visual Studio 2010 and no "Copy to OuputDirectory" was required on the CSV file in VS 2010.

    Why is it not the same way in VS 2012? Is that a known issue?

    Also I don't like the usage of .testsettings file as a long term solution to resolve this issie due to the following reason:
    Even though test settings file is added as a solution item, user has to manually choose the test settings file for the solution through : Test->Test Settings ->Select Test Settings.

    I guess the change made made with this is part of the .suo file and hence every developer has to do this to get the tests working.

    I haven't checked how it is going to work on the build machine

    Please correct me if I am wrong.

    Thanks,

    Sudheer  


    Monday, March 18, 2013 6:22 PM
  • Hello Sudheer,

    Since there is no wizard like in VS2010 used to set the datasource for test method in VS2012, we have to add the datasource attribute and DeploymentItem attribute for test method manually in VS2012. In this case we need consider the path of the deployed file to configure DeploymentItem attribute.

    I don’t think that it is an issue in VS2012. Configuring the appropriate DeploymentItem attribute based on the files located on different directories and “Copy to OutputDirectory” property can make tests successful on my side.

    If you want VS2012 to perform the same way with VS2010 and don't want to manually select the target test settings file, you could submit this feature request:

    http://visualstudio.uservoice.com/forums/121579-visual-studio

    The Visual Studio product team is listening to user voice there. You can send your ideas/suggestions there and people can vote.

    Thanks for your understanding.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    • Marked as answer by Sudheer Kumar Tuesday, March 19, 2013 11:45 PM
    Tuesday, March 19, 2013 1:50 AM
  • I got the core of the issue that in VS 2012, the path of the data file has to be relative to the project folder, where are in VS 2010, the path has to be relative to the solution folder. I have uploaded a working solution for VS 2012 w/o using the testsettings file. http://code.msdn.microsoft.com/Data-Drive-Unit-Tests-0ecd211e Thanks Amanda for your support in resolving this issue.
    Tuesday, March 19, 2013 11:44 PM
  • Hello,

    Glad to see this issue has been resolved and thank you for sharing your solutions & experience here. It will be very beneficial for other community members who have similar questions.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Wednesday, March 20, 2013 12:44 AM
  • I tried to download that solution and it does not work.  Says it is not published. 

    Also, downloads go away over time, can anyone post what the solution was?

    Tuesday, June 4, 2013 9:07 PM