locked
[DataSource] attribute on [TestMethod] evaluated before [ClassInitialize] method is run? RRS feed

  • Question

  • Hi.

    I have tests that were running fine in VS2010, but fail in VS2012. 

    The test class has a [ClassInitialize] method that basically builds an XML file that is used by the test methods in the class as a [DataSource].

    When I attempt to run these tests in VS2012, I get the following output:

    Test Outcome: Failed
    Test Duration: 0:00:00

    Result Message: The unit test adapter failed to connect to the data source or to read the data. For more information on troubleshooting this error, see "Troubleshooting Data-Driven Unit Tests" (http://go.microsoft.com/fwlink/?LinkId=62412) in the MSDN Library. Error details: Object reference not set to an instance of an object.

    I'v tried putting a breakpoint in the [ClassInitialize] method, but the it's never hit, unless I'm debugging a test method that doesn't have the [DataSource] attribute.

    Please advise.

    Sunday, October 28, 2012 12:51 PM

Answers

  • Hello Pat,

    According to your code, I think that maybe there is anything wrong with your connection string. You may specify the location of the XML file to check if it helps. You can refer to:

    [DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML", "|DataDirectory|\\CalendarHolidayTestData.xml", "TestTable", DataAccessMethod.Sequential)]

    In addition, if your xml file is not in the test project directory, I suggest that you can try to set it in the project directory.

    [ClassInitialize()] Use ClassInitialize to run code before you run the first test in the class.

    I think that ClassInitialize method should be called.  In order to confirm if the ClassInitialize method is called before test method, you can run your TestMethod_DoesntUserDataSource first to check if the test can run successfully.

    If it can run fine, the ClassInitialize method should be called before the test method TestMethod_UserDataSource.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us

    • Marked as answer by Amanda Zhu Friday, November 2, 2012 8:16 AM
    Tuesday, October 30, 2012 1:46 AM
  • Amanda,

    Thanks for inquiring and for all your help. I consider this question resolved.

    My original question has been answered: I was worried that VS2012's implementation had somehow changed in a way that would explain the symptoms. But it hasn't, so I have to look elsewhere.

    Like I said in my last post, I wasn't able to reproduce the issue with a small sample, so the problem must be in the actual tests. I have an idea as to where the problem area actually lies, and it's in my code.

    -Pat


    -Pat

    • Marked as answer by Pat Tremblay Friday, November 2, 2012 4:59 PM
    Friday, November 2, 2012 4:58 PM

All replies

  • Hi again,

    Some more details:

    - I've verified that I'm using the V11 UnitTestFramework assembly. I set the Copy Local property on the reference and checked the assembly version property in the bin\debug folder.

    - I opened a VS2012 Command Prompt and ran the tests from the command line using the command:

    mstest /testcontainer:ReportConfigValidationTests.dll

    and they run fine!


    Sunday, October 28, 2012 1:07 PM
  • Hello Pat,

    Now you can run your test using MSTest fine. Can you use data in XML file successfully?

    If so, now can you run the test in VS IDE fine?

    If you still can’t run the test, can you provide detailed code about classInitialize method and test method so that we can repro your issue? Also you can attach your test project, upload it to the sky driver, and then share the download link in your post.

    According to your description, if you do not use the DataSource attribute in the test method, I think that it is mostly possible to get the error.

    About how to use a data source for unit test in VS2012, please refer to this article:

    How to: Create a data-driven unit test

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us

    Monday, October 29, 2012 4:48 AM
  • Hi Amanda,

    To answer your questions:

    • Yes I can run the tests using mstest from the command line: the [ClassInitialize] method generates the XML file correctly and the [TestMethod] can use the data in the XML. I can even open the generated trx file in VS2012 and diagnose the test failures, if any.
    • I am still UNABLE to run the tests from Visual Studio. I still get the error that I explained in my first post.
    • I can't post the code as it is, since I don't have permission from my employer. I'll try to put together a small sample that reproduces the issue and post it.
    • Removing the [DataSource] attribute is not an option, since the test requires the data to run. But, if in the same [TestClass], I add a [TestMethod] that doesn't use the [DataSource], then I can debug my test fine and I can even put a breakpoint in the [ClassInitialize] method. This leads me to the conclusion that, when running the tests inside VS, the [DataSource] is evaluated first and when it fails, the [ClassInitialize] method is not even called. Hope this is clearer.

    Monday, October 29, 2012 10:00 PM
  • Hi,

    I've written a small test class that simulates what my real test class does (posted the code below) but I can't reproduce my issue: both tests below pass. So in a sense, this is good news, because it means the bug is somewhere in my code. 

    So thanks for your help. Pushing me to come up with a small repro actually pointed me in the right direction.

    -Pat


    using System;
    using System.Linq;
    using System.Xml;
    using System.Xml.Linq;
    using Microsoft.VisualStudio.TestTools.UnitTesting;
    using System.IO;
    
    namespace UnitTestProject3
    {
        [TestClass]
        public class UnitTest1
        {
            public TestContext TestContext { get; set; }
    
            const string DataCatalogFileName = "@TestData.xml";
            static bool ClassInitializeWasCalled = false;
    
            [ClassInitialize]
            public static void ClassInitialize(TestContext context)
            {
                var dataName = new[] { "DATA1", "DATA2", "DATA3" };
    
                var rowNumber = 0;
                var dataCatalog = new XElement("dataCatalog");
                foreach (var name in dataName)
                {
                    var element = new XElement("testData",
                                    new XElement("name", name),
                                    new XElement("rowNumber", rowNumber++));
                    dataCatalog.Add(element);
                };
    
                // Write out the file
                File.WriteAllText(DataCatalogFileName, dataCatalog.ToString());
                ClassInitializeWasCalled = true;
            }
    
            [TestMethod]
            [DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML", DataCatalogFileName, "testData", DataAccessMethod.Sequential)]
            public void TestMethod_UsesDataSource()
            {
                var name = TestContext.DataRow["name"].ToString();
                Assert.IsTrue(name.StartsWith("DATA"));
            }
    
            [TestMethod]
            public void TestMethod_DoesntUseDataSource()
            {
                // This test should succeed
                Assert.IsTrue(ClassInitializeWasCalled);
            }
        }
    }
    

    Tuesday, October 30, 2012 12:11 AM
  • Hello Pat,

    According to your code, I think that maybe there is anything wrong with your connection string. You may specify the location of the XML file to check if it helps. You can refer to:

    [DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML", "|DataDirectory|\\CalendarHolidayTestData.xml", "TestTable", DataAccessMethod.Sequential)]

    In addition, if your xml file is not in the test project directory, I suggest that you can try to set it in the project directory.

    [ClassInitialize()] Use ClassInitialize to run code before you run the first test in the class.

    I think that ClassInitialize method should be called.  In order to confirm if the ClassInitialize method is called before test method, you can run your TestMethod_DoesntUserDataSource first to check if the test can run successfully.

    If it can run fine, the ClassInitialize method should be called before the test method TestMethod_UserDataSource.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us

    • Marked as answer by Amanda Zhu Friday, November 2, 2012 8:16 AM
    Tuesday, October 30, 2012 1:46 AM
  • Hello Pat,

    Any update? Could you get useful information from our reply?

    Would you mind letting us know the result of the suggestion?

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us

    Thursday, November 1, 2012 5:51 AM
  • Amanda,

    Thanks for inquiring and for all your help. I consider this question resolved.

    My original question has been answered: I was worried that VS2012's implementation had somehow changed in a way that would explain the symptoms. But it hasn't, so I have to look elsewhere.

    Like I said in my last post, I wasn't able to reproduce the issue with a small sample, so the problem must be in the actual tests. I have an idea as to where the problem area actually lies, and it's in my code.

    -Pat


    -Pat

    • Marked as answer by Pat Tremblay Friday, November 2, 2012 4:59 PM
    Friday, November 2, 2012 4:58 PM