locked
Tests that use Fakes/Shims fail on server, but succeed in local RRS feed

  • Question

  • Tests that use Fakes/Shims fail on build server, but succeed in local.

    This following error is gone when I remove the test that uses shims. Do I need to do any specific configuration on build server?

    On a side note, I'm using shims in my specflow test cases.

    Run has the following issue(s):
    One of the background threads threw exception: 
    System.NullReferenceException: Object reference not set to an instance of an object.
       at lambda_method(Closure , IContextManager )
       at TechTalk.SpecFlow.Bindings.BindingInvoker.InvokeBinding(IBinding binding, IContextManager contextManager, Object[] arguments, ITestTracer testTracer, TimeSpan& duration)
       at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.FireEvents(HookType bindingEvent)
       at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.OnTestRunEnd()
       at TechTalk.SpecFlow.TestRunner.OnTestRunEnd()
       at TechTalk.SpecFlow.TestRunnerManager.OnTestRunnerEnd()
       at TechTalk.SpecFlow.TestRunnerManager.<InitializeBindingRegistry>b__12_0(Object <sender>, EventArgs <e>)


    Friday, September 1, 2017 7:10 PM

All replies

  • Hi friend,

    >>Tests that use Fakes/Shims fail on build server, but succeed in local.

    Based on the info you posted, it works locally, you probably have a problem in your build server settings, please refer to the doc:

    Prepar your build server

     

    Best regards,

    Fletcher


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.



    • Edited by Fletch Zhou Monday, September 4, 2017 8:10 AM
    Monday, September 4, 2017 8:08 AM
  • Hi clramesh 

    based on the error that i  can see. you have a null exception on an object. Try go figure out what might cause this null reference. It could be that your build server has different configuration so when your test runs at the build server it fails and gives a null reference. 

    Maybe you can provide us the unit test so we can have a look

    friendly regards

    Laurens



    • Edited by laurens vdb Monday, September 4, 2017 9:09 AM
    Monday, September 4, 2017 9:08 AM
  • Thanks for your reply Fletch, I have build server setup and running for several years. 

    Link you have provided says, build server need VS ultimate version for faked assemblies. We have VS enterprise edition installed, will it not work with it?

    Tuesday, September 5, 2017 1:20 PM
  • Thanks Laurens for your reply. As you know its not easy to debug things on build servers, I'm wondering if what could be the special config settings needed for Microsoft fakes/shims. 
    Tuesday, September 5, 2017 1:22 PM
  • I found out the error is

    Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationException: Failed to resolve profiler path from COR_PROFILER_PATH and COR_PROFILER environment variables.

    Not sure how to fix it though.

    Tuesday, September 5, 2017 2:34 PM
  • Hi clramesh

    Specify the Test Runner to be Visual Studio Test Runner in the build server

     You can find this by editing the build definition > Under Automated Tests, open the Test Source dialog > Under Test runner, select Visual Studio Test Runner.


    • Edited by laurens vdb Tuesday, September 5, 2017 2:36 PM
    Tuesday, September 5, 2017 2:36 PM
  • Finally I'm able to resolve the issue by using the vstest.console.exe instead of mstest.exe. 

    The following article mentioned that mstest doesnt support fakes. Only vstest supports it. 

    https://msdn.microsoft.com/en-us/library/ms253138(v=vs.110)

    Thanks!

    • Proposed as answer by Fletch Zhou Thursday, September 7, 2017 1:36 AM
    Wednesday, September 6, 2017 1:50 PM
  • Hi clramesh,

    Thanks for sharing us the solution.

    You could mark you own reply as answer if it's ok for you.

    Have a nice day:)

     

    Best regards,

    Fletcher


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.


    • Edited by Fletch Zhou Thursday, September 7, 2017 1:38 AM
    Thursday, September 7, 2017 1:38 AM