locked
MS Test with #line directive containing invalid file name prevents test from running when right-clicking -> "Run Tests" RRS feed

  • Question

  • I recently installed Visual Studio 2013 Update 4. Previously, I was using Visual Studio 2010 (and this is still installed on my system).

    I've discovered a behavior that is inconsistent between 2010 and 2013. If an MS Test method contains a #line 1 "SomeFile.foo" line directive where SomeFile.foo is not a valid file (or it is a valid file just not a C# file), then right clicking inside the test method and clicking "Run Tests" in VS 2013 won't run the test. In fact, nothing happens at all after the project is built. No error. No warning. Nothing.

    Sample Code:

    using System;
    using System.Text;
    using System.Collections.Generic;
    using System.Linq;
    using Microsoft.VisualStudio.TestTools.UnitTesting;
    
    namespace MsTestBugDemo
    {
        [TestClass]
        public class UnitTest1
        {
            [TestMethod]
            public void TestMethod1()
            {
    #line 1 "UnitTest1.foo"
                Assert.IsTrue(true);
            }
        }
    }



    In the example above, I used the "UnitTest1.cs" file generated by Visual Studio when a new test project is created. The only thing I added as the #line directive containing an invalid file name.

    In Visual Studio 2010, right clicking on the test method above and clicking "Run Tests" will run the "TestMethod1" test.

    In Visual Studio 2013, right clicking on the test method above and clicking "Run Tests" will build the project, but that's it.

    I created a GitHub project with a bare bones example: https://github.com/gburghardt/MsTestBugDemo

    I'm not sure if this is simply a change in the behavior, or a bug in the test runner introduced since VS 2010 came out.

    As some background, I initially went down this rabbit hole because I could no longer run single SpecFlow scenarios by right clicking on the scenario in the feature file, and clicking "Run Specflow Scenarios".

    GitHub issue: https://github.com/techtalk/SpecFlow/issues/452

    The bug outlined in this post might be related to the SpecFlow bug, but I'm not sure. I just know I was able to create a very simple test case without SpecFlow and thought I'd post it here.

    PS: Sorry I couldn't post links. Apparently my account still needs verification.


    • Edited by Greg Burghardt Thursday, June 18, 2015 12:22 PM Inserted links
    Tuesday, June 16, 2015 6:01 PM

Answers

  • Hi Tina-Shi,

    Sorry about that. I did submit feedback on Connect, but forgot to include the link:

    https://connect.microsoft.com/VisualStudio/feedback/details/1449822/line-directive-in-ms-test-unit-test-referencing-file-name-prevents-test-from-running-when-right-clicking-run-tests-from-source-code

    • Marked as answer by Tina-Shi Thursday, June 25, 2015 9:41 AM
    Monday, June 22, 2015 12:18 PM

All replies

  • Hi Greg,

    Thank you for posting in MSDN forum.

    According the unit test code as you provided, I tried to new a simple unit test project separately in VS2013 and VS2010 and then copy your code into unit test method, run the unit test method by right-click test method->select Run Tests, but I found that I get same result with yours.

    The VS2010 works fine for this unit test include the #line 1 "UnitTest1.foo" invalid file name, the Run tests option is not work for this unit test include the  #line 1 "UnitTest1.foo"  invalid file name in the VS2013.

    However, I try to run the unit test by selecting the Selected Run tests option in the Test Explorer window in the VS2013, I found that the unit test run successfully in the VS2013.

    So please try to run it from this Text Explorer and then check this issue again. https://msdn.microsoft.com/en-us/library/hh270865.aspx

    Note: the Test Explorer windows is a new feature in the VS2013.

    In addition, as you said about this SpecFlow, I did some research about it from this link:

    https://visualstudiogallery.msdn.microsoft.com/90ac3587-7466-4155-b591-2cd4cc4401bc

    I found that the SpecFlow is a extension too for this VS2013, so please start a discussion to the Q AND A in the above site, you will get better respond.

    Thanks for your understanding!

    Best Regards,


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Thursday, June 18, 2015 2:49 AM
  • Hi Tina-Shi,

    It runs fine from the Test Explorer, but I find it much easier to just go straight to the unit test file. Especially if I'm working on that particular test. Furthermore, I think this bug might be related to a bug I discovered in SpecFlow when attempting to run a single Scenario (which maps to an MS Test unit test).

    Thursday, June 18, 2015 12:20 PM
  • Hi Greg,

    >>but I find it much easier to just go straight to the unit test file. Especially if I'm working on that particular test.

    Generally, I know that Run Selected Tests is used to run the particular test in Test Explorer window after you build your test solution/project.

    So I think that it is default that Run tests option is not work for this unit test include the  #line 1 "UnitTest1.foo"  invalid file name in the VS2013.

    If you still want to this feature request, I suggest you could submit a feedback to Microsoft Connect feedback portal:http://connect.microsoft.com/VisualStudio/feedback/CreateFeedback.aspx, Microsoft engineers will evaluate them seriously.

    If you submit it, you could share me the link here, I will help you vote it.

    Or I suggest you could try to start a discussion to the SpecFlow for Visual Studio 2013 as I pervious said, maybe you will get better respond for this SpecFlow.

    Thanks for your understanding.

    Best Regards,


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Friday, June 19, 2015 1:56 AM
  • This is not something that SpecFlow can fix. My code example was a little misleading.

    SpecFlow is a .NET implementation of the Gherkin language (https://github.com/cucumber/cucumber/wiki/Gherkin). It is a way to write behavior driven tests. They get transformed to MS Test unit tests. SpecFlow uses line directives in the unit tests so when a test fails, you get the file name and line number of a feature file, not the underlying unit test.

    SomethingImportant.feature

    Feature: Something important
    
    Scenario: Important things should be done
        Given something
        When something
        Then something

    Gets translated to SomethingImportant.feature.cs

    [TestClass]
    public partial class SomethingImportantFeature
    {
    	private static TechTalk.SpecFlow.ITestRunner testRunner;
    
    #line 1 "SomethingImportant.feature"
    
    	[TestMethod]
    	public virtual void ImportantThingsShouldBeDone()
    	{
    #line 4
    		testRunner.Given("Something important");
    #line 5
    		testRunner.When("Something important");
    #line 6
    		testRunner.Then("Something important");
    #line hidden
    	}
    }

    The SomethingImportant.feature file is a valid file, it's just not a C# file. The behavior I originally outlined in this thread is exhibited in this case as well. This leads me to believe that SpecFlow can't really fix this. This is a bug or change in behavior for Visual Studio between 2010 and 2013.

    I'll try submitting feedback.

    Thanks.

    Friday, June 19, 2015 12:26 PM
  • Hi Greg,

    Thanks for your reply.

    Could you please tell me if you submit the feedback to the connect?

    If you have submitted, please share me the link here so that we will help you vote it.

    Thanks,


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Monday, June 22, 2015 3:18 AM
  • Hi Tina-Shi,

    Sorry about that. I did submit feedback on Connect, but forgot to include the link:

    https://connect.microsoft.com/VisualStudio/feedback/details/1449822/line-directive-in-ms-test-unit-test-referencing-file-name-prevents-test-from-running-when-right-clicking-run-tests-from-source-code

    • Marked as answer by Tina-Shi Thursday, June 25, 2015 9:41 AM
    Monday, June 22, 2015 12:18 PM