locked
Unit tests work when using the "Run test after build setting". RRS feed

  • Question

  • I've just migrated the solution and our unit tests from VS2010 to VS2012. We have replaced the Moles framework with the new version called Fakes that's shipped with VS2012. 

    When i use the "Test-> Test Settings -> Run test after build" setting all 132 test are passed. But when i then press the "Run All" button in the Test Explorer the 46 test that use Fakes fail with a ShimNotSupportedException exception.

    Full info as follows:

    Test Name: ErrorHandle_ForSuccessfulOperationAndNoExceptionLevelSpecified_ShouldReturnTrueTest FullName: DLCInbound.Business.Test.IncomingOrder.ErrorHandlingTest.ErrorHandle_ForSuccessfulOperationAndNoExceptionLevelSpecified_ShouldReturnTrueTest Source: c:\tfs\DLCInbound\Main\Src\DLCInbound.Business.Test\IncomingOrder\ErrorHandlingTest.cs : line 28Test Outcome: FailedTest Duration: 0:00:00,001957Result Message:Test method DLCInbound.Business.Test.IncomingOrder.ErrorHandlingTest.ErrorHandle_ForSuccessfulOperationAndNoExceptionLevelSpecified_ShouldReturnTrue threw exception: Microsoft.QualityTools.Testing.Fakes.Shims.ShimNotSupportedException: DLCInbound.Business.SystemEventBLResult StackTrace: at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.InvokeEvent[T](T value, Action`1 eh)    at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.OnAttachedUnsupportedMethod(MethodBase method)    at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.CheckInstrumentation(MethodBase method)    at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.InternalAttachDetour(Object optionalReceiver, MethodBase method, Delegate detourDelegate)    at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.AttachDetour(Object optionalReceiver, MethodBase method, Delegate detourDelegate)    at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.SetShimMethod(Delegate optionalStub, Object optionalReceiver, MethodBase method)    at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.SetShim(Delegate optionalStub, Type receiverType, Object optionalReceiver, String name, ShimBinding flags, Type returnType, Type[] parameterTypes)    at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.SetShimPublicStatic(Delegate optionalStub, Type receiverType, String name, Type returnType, Type[] parameterTypes)    at DLCInbound.Business.Fakes.ShimSystemEventBL.set_LogSystemEventStringStringStringStringGuidStringString(Action`7 value) in c:\tfs\DLCInbound\Main\Src\DLCInbound.Business.Test\obj\Debug\Fakes\dlcib\f.cs:line 0    at DLCInbound.Business.Test.IncomingOrder.ErrorHandlingTest.MockSystemEventBL() in c:\tfs\DLCInbound\Main\Src\DLCInbound.Business.Test\IncomingOrder\ErrorHandlingTest.cs:line 245    at DLCInbound.Business.Test.IncomingOrder.ErrorHandlingTest.ErrorHandle_ForSuccessfulOperationAndNoExceptionLevelSpecified_ShouldReturnTrue() in c:\tfs\DLCInbound\Main\Src\DLCInbound.Business.Test\IncomingOrder\ErrorHandlingTest.cs:line 31

    So basically, tests only seems to succeed when I run them as part of the build. Does anyone have any idea of what could cause this?

    Thanks in advance

    Johan Nordlinder

    UPDATE 1

    I've noticed that if I run the test one by one they all pass. Only when running more than ~3 at the same time they break. But why doesn't it break right after the build when they also all run at the same time? This was never any problem before when we used the Moles framework.

    UPDATE 2

    I've noticed that it's only when the tests fail I get the following message in the Test Output panel during the build, it would be usefull if it was possible to get more information about the error....

    ========== Discover test finished: 132 found (0:00:04,50045) ==========

    ------ Run test started ------

    Failed to configure settings for runsettings plugin 'Fakes' as it threw following exception:

    'Object reference not set to an instance of an object.'

    Please contact the plugin author.

    Wednesday, March 27, 2013 11:42 AM

Answers

  • UPDATE 3

    I had Visual Studio 2012 Update 1 installed, when I also installed the Update 2 the problem went away. I also confirmed with a colleague that only had this problem with the Update 1, without any updates or with Update 2 applied there was no problem.

    Wednesday, April 24, 2013 10:39 AM

All replies

  • Hi Johan,

    I am trying to involve someone familiar with this topic to further look at this issue. There might be some time delay. Appreciate your patience.

    Best Regards,


    Jack Zhai [MSFT]
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Thursday, March 28, 2013 10:39 AM
  • UPDATE 3

    I had Visual Studio 2012 Update 1 installed, when I also installed the Update 2 the problem went away. I also confirmed with a colleague that only had this problem with the Update 1, without any updates or with Update 2 applied there was no problem.

    Wednesday, April 24, 2013 10:39 AM