locked
VS 2015 Update 2 with a .runsettings file and parameters RRS feed

  • Question

  • After installing Update 2 that was published on Mar 30th, 2016, I am running into an issue with VSTest and TestRunParameters.  It was working with Update 1, but now the TestContext does not contain any properties that are provided inside the .runsettings file even though I set the file inside Test -> Test Settings -> Select Test Settings File

    Here is my runsettings file:

    <?xml version="1.0" encoding="utf-8"?>
    <RunSettings>
      <!-- Parameters used by tests at runtime -->
      <TestRunParameters>
        <Parameter name="webAppUrl" value="https://localhost/" />
        <Parameter name="Username" value="User1" />
        <Parameter name="Password" value="Password" />
        <Parameter name="RemoteBrowser" value="Chrome" />
        <Parameter name="APIUrl" value="https://localhost/API" />
      </TestRunParameters>
    </RunSettings>
    

    Here is my code to grab the value:

    return context.Properties["Username"].ToString();

    Is this a new bug?

    Wednesday, April 20, 2016 4:54 PM

All replies

  • Hi JakeFe,

    Is it the unit test project or coded UI test or others? I have the VS2015 with update 1 and update 2 Environment, to help you repro this issue as soon as possible, could you share us a simple sample using one drive? So we could really repro it in our side using the same Environment as yours.

    Please attach your Visual Studio project, you can upload it to the one drive, and then share the download link in your post. Please also share us the different results with update 1 and update2 using screen shots in your reply. If I could repro this  issue, I will help you report this issue to the product team.

    Sincerely,

    Jack

    Update:

    I create a simple unit test using this runsettings file in my VS2015 Enterprise version with update 2.

    [TestClass]
        public class UnitTest1
        {
            public TestContext TestContext
            {
                get
                {
                    return testContextInstance;
                }
                set
                {
                    testContextInstance = value;
                }
            }
            private TestContext testContextInstance;
    
    
    
            [TestMethod]
            public void TestMethod1()
            {
                string a = TestContext.Properties["Username"].ToString();
    
                Console.WriteLine(a);
            }
        }

    I could get the output value in my side, so it seems that it still works well in the VS2015 update 2.

    If possible, you could test it using this sample in your side, if it doesn't work, we would think about the VS IDE, but if it works well, I think we would think about the project code itself.

    Best Regards,

    Jack


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Thursday, April 21, 2016 11:39 AM
  • Hi JakeFe,

    I update my reply after a simple testing, would you mind letting us know the latest information in your side?

    Best Regards,

    Jack


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Thursday, May 12, 2016 10:39 AM
  • Hi Jack,

    I have exactly similar issue , I am using VS2015 with update 1 & 2,and i did test it works fine with Unit Test but not with Coded UI test.

    Regards,

    SS

    Thursday, May 12, 2016 2:54 PM
  • Hi SS,

    The .runsettings file is used to configure unit tests for some purposes. I think it would have a limitation for the coded UI tests.

    https://msdn.microsoft.com/en-us/library/jj635153.aspx?f=255&MSPPError=-2147217396

    Best Regards,

    Jack


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Saturday, May 14, 2016 8:20 AM
  • Hi Jack,

    The .runsettings file some parameters do work with coded UI tests, i have testing that with changing the result directory in runconfiguration.

    <RunConfiguration>
        <ResultsDirectory>.\TestResults</ResultsDirectory>
     </RunConfiguration>

    But it seems not to work with TestRunParameters

    <TestRunParameters>
        <Parameter name="webAppUrl" value="http://localhost" />
        <Parameter name="webAppUserName" value="Admin" />
        <Parameter name="webAppPassword" value="Password" />
    </TestRunParameters>

    Although i did read some posts where it was mentioned that it did work fine with 2013 with after installing update 5, so i was wondering whether it will work with 2015.

    Thanks,

    SS

    Tuesday, May 17, 2016 10:55 AM
  • Hi SS,

    I just test it using a simple coded UI test project in VS2013 with update 5 and VS215 with update 2, but it seems that it doesn't work in my side.

    The same sample like my previous reply, I will get the error "Test method UnitTestProject1.CodedUITest1.CodedUITestMethod1 threw exception:
    System.NullReferenceException: Object reference not set to an instance of an object." in coded UI test project, but it really works well for unit tests.

    Reference:

    http://stackoverflow.com/questions/32597776/testcontext-testrunparameters-in-coded-ui-test

    It would have a limitation for TestRunParameter.

    Best Regards,

    Jack


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Wednesday, May 18, 2016 4:30 AM
  • Hello Jack,

    I'm following up on this as I think I have pretty easy repro steps that illustrates the original issue from this thread.  I've included a sample VS solution at this OneDrive location
    https://onedrive.live.com/?authkey=%21AEW6DFQela29s9M&id=A1218136586B32F1%2135173&cid=A1218136586B32F1

    Repro steps:

    1. In Visual Studio 2015 Update 1, open the supplied solution.

    2. Select the sample1.runsettings file via Test -> Test Settings

    3. Build and run all the tests (they'll simply pass).  In this case there are only 2 tests and each just correctly outputs the value of a parameter from the currently selected runsettings file.

    4. Select the sample2.runsettings file via Test -> Test Settings

    5. Now run just one of the tests (i.e. TestMethod1).  Note that the output console value correctly displays the parameter value from sample2.runsettings.

    6. Now open Visual Studio 2015 with Update 2 and open the same supplied solution.

    7. Perform steps 2 through 5 in VS 2015 with Update 2.  Note now that step 5 will fail because the output console value is incorrect as it still shows the value from sample1.runsettings instead of the value from sample2.runsettings.

    Can you confirm you can reproduce the issue and that it's a valid bug?  In our organization, we rely pretty heavily on switching testsettings and runsettings files to switch tests to run against different environments via this mechanism.

    Thanks for any feedback you have.

    Donnie



    • Edited by DWick Wednesday, May 18, 2016 10:35 PM
    Wednesday, May 18, 2016 10:34 PM
  • Hi Donnie,

    Maybe you could put the two class in the same .cs file.

    For example, change your unittest2.cs file:

    [TestClass]
        public class UnitTest2 : TestBase
    
        // public class UnitTest2
        {
            [TestMethod]
            public void TestMethod2()
            {
                string simpleParameter = TestContext.Properties["TestParameter"].ToString();
                Console.WriteLine("Test Parameter Vvalue: {0}", simpleParameter);
                Assert.IsFalse(string.IsNullOrEmpty(simpleParameter));
            }
    
        }
    
    
        [TestClass]
        public class TestBase
        {
            public TestBase()
            {
                //
                // TODO: Add constructor logic here
                //
            }
    
            private TestContext testContextInstance;
    
            /// <summary>
            ///Gets or sets the test context which provides
            ///information about and functionality for the current test run.
            ///</summary>
            public TestContext TestContext
            {
                get
                {
                    return testContextInstance;
                }
                set
                {
                    testContextInstance = value;
                }
            }
    
        }

    Best Regards,

    Jack


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Thursday, May 19, 2016 10:21 AM
  • Hello Jack,

    I tried your suggestion and I still see the same issue called out in my repro steps.  I guess I'm not sure how combining classes would solve the issue, but I tried and still see the same bug.

    1. Can you confirm this is a bug with VS 2015 Update 2?
    2. What's the best course of action to submit this as a bug to the VS Team to be picked up as a fix for an upcoming update?

    Thanks for any feedback,

    Donnie


    Friday, May 20, 2016 9:06 PM
  • Hi Donnie,

    Thanks for your friendly response.

    I think I misunderstood this issue, firstly, I think you couldn't get the output value. But it seems that the real issue is that it shares the same output value using the two different .runsettings file.

    Please clean and rebuild the solution before you run your test again, how about the result?

    Actually if I change the runsettings file, and right click the test method (passed before) and select "run selected test", it really doesn't output the correct value, but I feel that the test doesn't run since it would output the correct value if I clean and rebuild solution, and then run it.

    So for this issue, if possible, please clean and rebuild  your solution, run it again.

    Of course, you can submit this feedback to Microsoft Connect feedback portal: http://connect.microsoft.com/VisualStudio/feedback/CreateFeedback.aspx,Microsoft engineers will evaluate them seriously.

    Best Regards,

    Jack


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Saturday, May 21, 2016 6:02 AM