none
Fakes framework: ShimNotSupportedException?

    Question

  • Hi,

    I've been using the fakes isolation framework for testing some of our production code across several assemblies.

    Today I added new fakes for a different assembly, but when I tried to assign a delegate to one of the instance methods it I receive a "ShimNotSupportedException". (Note that I previously used the Moles framework successfully with the same assembly).

    Could someone explain why I get this exception, and what the pre-requisites are for an assembly to be successfully 'faked'?

    Thanks,

    Nick

    Friday, April 13, 2012 2:28 PM

All replies

  • Hello Nick,

    I am trying to involve someone familiar with this topic to further look at this issue.

    Thanks.


    Vicky Song [MSFT]
    MSDN Community Support | Feedback to us

    Tuesday, April 17, 2012 7:30 AM
    Moderator
  • Hi Vicky,

    Thanks- here is an example of an assembly that causes the issue:

    Example assembly

    Here is how to repro the ShimNotSupportedException:

        [TestMethod]
        public void TestMethod1()
        {
          using (ShimsContext.Create())
          {
            ShimMap.AllInstances.LayersGet = (instance)=> null;
          }
        }

    Regards,

    Nick

    Tuesday, April 17, 2012 9:33 AM
  • Hey Vicky- did you discover any more information on this exception? 

    Thanks,

    Nick

    Tuesday, April 24, 2012 10:38 AM
  • Hello Nick,

    I've got someone looking at your issue. Thanks for reporting it.


    -- Peter Provost Sr. Program Manager Visual Studio ALM Microsoft Corporation

    Tuesday, April 24, 2012 2:03 PM
  • Hi,

    I'm seeing this issue as well, and though I know you probably don't have a fix for this yet, I was wondering if you could tell us what triggers this? Looking for potential workarounds.

    Regards,

    Stephen.

    EDIT: I know what is causing this problem in our case. Although the decision happens in unmanaged code, so I can't be 100% sure, it seems the Shims functionality expects to be the only thing instrumenting the assembly. If anything else is already instrumenting the assembly, Fakes will refuse to create the Shim. In our case, it was because we had enabled Code Coverage, which also uses instrumentation. 

    • Edited by wild_ginger Friday, May 18, 2012 8:27 AM
    • Proposed as answer by wild_ginger Friday, May 18, 2012 8:28 AM
    Friday, May 11, 2012 12:21 PM
  • EDIT: I know what is causing this problem in our case. Although the decision happens in unmanaged code, so I can't be 100% sure, it seems the Shims functionality expects to be the only thing instrumenting the assembly. If anything else is already instrumenting the assembly, Fakes will refuse to create the Shim. In our case, it was because we had enabled Code Coverage, which also uses instrumentation. 

    This seems very unlikely to me, but might help us track down the problem. Shims and Dev11 Code Coverage use the same instrumentation technology so they work together quite well.

    How were you enabling CC? Are you using a .testsettings file?


    -- Peter Provost Sr. Program Manager Visual Studio ALM Microsoft Corporation

    Friday, May 18, 2012 3:29 PM
  • Yes, that's correct, in the Local.testsettings. I apologise if it turns out to be a wild goose chase, but the ShimNotSupported exceptions disappear as soon as I turn Code Coverage off, and re-appear when I turn it back on again.

    Regards,

    Stephen.

    • Proposed as answer by Thales Fu Wednesday, September 26, 2012 6:50 AM
    Monday, May 21, 2012 7:42 AM
  • Hello,

    I'll need some data on this.

    1. In your testsettings, which collector is enabled: 'Code Coverage (Visual Studio 2010)' or 'Code Coverage' or something different?

    2. If you remove your testsettings file, and then run code coverage via the Test -> Analyze Code Coverage, do the tests run?

    Thanks

    Sudhakar [MSFT]

    Tuesday, May 22, 2012 1:29 PM
  • Hi, I should say that the tests always run, they just don't pass because of thrown exceptions.

    1. I've had "Code Coverage" enabled. If I enable "Code Coverage (Visual Studio 2010)" instead, the tests don't generate ShimNotSupported exceptions.

    2. Yes:

    • If I manually run code coverage without a testsettings file, the tests pass and code coverage is correctly analyzed.
    • If I manually run code coverage with a testsettings file, but with no data diagnostic adapters enabled, the tests pass and code coverage is correctly analyzed.
    • If I manually run code coverage with a testsettings file, but with "Code Coverage (Visual Studio 2010)" enabled, the tests pass and code coverage is correctly analyzed.
    • If I manually run code coverage with a testsettings file, but with "Code Coverage" enabled, the tests fail with a ShimNotSupported exception and code coverage is not correctly analyzed.

    I'll attach the relevant portion of the stack trace from one test (starting from my Shim'ed class, edited for confidentiality purposes):

    Result StackTrace: at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.InvokeEvent[T](T value, Action`1 eh) at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.CheckInstrumentation(MethodBase method) at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.InternalAttachDetour(Object optionalReceiver, MethodBase method, Delegate detourDelegate) at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.AttachDetour(Object optionalReceiver, MethodBase method, Delegate detourDelegate) at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.SetShim(Delegate optionalStub, Type receiverType, Object optionalReceiver, String name, ShimBinding flags, Type returnType, Type[] parameterTypes) at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.SetShimPublicInstance(Delegate optionalStub, Type receiverType, Object optionalReceiver, String name, Type returnType, Type[] parameterTypes) at MyCompany.MyProject.Fakes.ShimMyClass.set_MyPropertyGet(Func`1 value)



    Thursday, May 24, 2012 7:33 AM
  • Thanks. This confirms my hypothesis. VS 11 code coverage is not meant to be used as a data collector in testsettings. It will not show up in testsettings in the next VS 11 public release (RC) onwards. It is to be invoked from the Unit Test Explorer window or from the direct VS menu options.

    So please deselect all code coverage collectors from testsettings, and invoke code coverage from Unit test explorer or VS Test menu.

    I would also suggest that you not use the .testsettings file with Fakes & Shims project, unless you actually need it. .testsettings is a pre-dev11 construct, and is primarily intended to be used with dev10 projects . There are some aspects of testsettings that are not supported with the dev11 unit testing enhancements.

    Friday, May 25, 2012 8:29 AM
  • Hi,

    Thanks for that. I've passed that information around internally.

    I've encountered an additional way to generate a ShimNotSupported exception, relating to debugging. Should I submit that to Connect?

    Regards,

    Stephen.

    Monday, May 28, 2012 9:37 AM
  • Hi,

    Thanks for that. I've passed that information around internally.

    I've encountered an additional way to generate a ShimNotSupported exception, relating to debugging. Should I submit that to Connect?

    Regards,

    Stephen.

    Either way. You can tell me about it here and I can let you know whether it is a bug or not, or you can file it on Connect and we will look at it there.

    -- Peter Provost Sr. Program Manager Visual Studio ALM Microsoft Corporation

    Monday, May 28, 2012 6:20 PM
  • Peter, 

      I am having the same issue, although I'm running the tests through NCover for code coverage.  Runs fine in VS2012 with the built in code coverage , but throws the ShimNotSupportedException when run through NCover.  

    Simple test:

    [Fact]
    public void DirectoryDeleteThrowsException()
    {
    using (ShimsContext.Create())
    {
    System.IO.Fakes.ShimDirectoryInfo.AllInstances.DeleteBoolean = (DirectoryInfo di, bool val) => {throw new IOException();};
    Assert.DoesNotThrow(() => Utilities.CleanApplicationFolders());
    }
    }

    throws:

     

    Utilities.UnitTests.UtilitiesUnitTests.DeleteThrowsException [FAIL]
       Microsoft.QualityTools.Testing.Fakes.Shims.ShimNotSupportedException : System.IO.DirectoryInfo
       Stack Trace:
          at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.InvokeEvent[T](T value, Action`1 eh)
          at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.OnAttachedUnsupportedMethod(MethodBase method)
          at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.CheckInstrumentation(MethodBase method)
          at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.InternalAttachDetour(Object optionalReceiver, MethodBase method, Delegate detourDelegate)
          at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.AttachDetour(Object optionalReceiver, MethodBase method, Delegate detourDelegate)
          at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.SetShimMethod(Delegate optionalStub, Object optionalReceiver, MethodBase method)
          at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.SetShim(Delegate optionalStub, Type receiverType, Object optionalReceiver, String name, ShimBinding flags, Type returnType, Type[] parameterTypes)
          at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.SetShimPublicInstance(Delegate optionalStub, Type receiverType, Object optionalReceiver, String name, Type returnType, Type[] parameterTypes)
          at System.IO.Fakes.ShimDirectoryInfo.AllInstances.set_DeleteBoolean(Action`2 value)

    I am going to email NCover about the issue since it seems specifically related to their product, but I figured I would also post the information here for the MS team.  

    Tuesday, June 12, 2012 9:50 PM
  • Hi Peter,

    I'm getting a ShimNotSupportedException trying to fake AsEnumerable and creating Fakes assembly for System.Core (VS 2012 RC):

            static void Main(string[] args)
            {
                var strings = new List<string>().AsEnumerable();
                using (ShimsContext.Create())
                {
                    ShimEnumerable.AsEnumerableOf1IEnumerableOfM0<string>(source => strings);
                    var z = new List<string>().AsEnumerable();
                }
            }

    Is System.Core special in any way?

    Thanks

    Voffka

    Saturday, June 16, 2012 5:08 PM
  • Hi Peter,

    I'm getting a ShimNotSupportedException trying to fake AsEnumerable and creating Fakes assembly for System.Core (VS 2012 RC):

            static void Main(string[] args)
            {
                var strings = new List<string>().AsEnumerable();
                using (ShimsContext.Create())
                {
                    ShimEnumerable.AsEnumerableOf1IEnumerableOfM0<string>(source => strings);
                    var z = new List<string>().AsEnumerable();
                }
            }

    Is System.Core special in any way?

    Thanks

    Voffka

    You can't use Shims from a Main routine. Shims requires a special profiler to be used that is setup by the Unit Test runner, so they can only be used within a unit test that runs via a plugin to the VS 2012 test runner.

    What are you trying to do with Shims in this example?

    --Peter


    -- Peter Provost :: Sr. Program Manager Lead :: Visual Studio ALM

    Saturday, June 16, 2012 7:14 PM
  • Peter, 

      I am having the same issue, although I'm running the tests through NCover for code coverage.  Runs fine in VS2012 with the built in code coverage , but throws the ShimNotSupportedException when run through NCover.  

    Simple test:

    [Fact]
    public void DirectoryDeleteThrowsException()
    {
    using (ShimsContext.Create())
    {
    System.IO.Fakes.ShimDirectoryInfo.AllInstances.DeleteBoolean = (DirectoryInfo di, bool val) => {throw new IOException();};
    Assert.DoesNotThrow(() => Utilities.CleanApplicationFolders());
    }
    }

    throws:

     

    Utilities.UnitTests.UtilitiesUnitTests.DeleteThrowsException [FAIL]
       Microsoft.QualityTools.Testing.Fakes.Shims.ShimNotSupportedException : System.IO.DirectoryInfo
       Stack Trace:
          at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.InvokeEvent[T](T value, Action`1 eh)
          at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.OnAttachedUnsupportedMethod(MethodBase method)
          at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.CheckInstrumentation(MethodBase method)
          at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.InternalAttachDetour(Object optionalReceiver, MethodBase method, Delegate detourDelegate)
          at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.AttachDetour(Object optionalReceiver, MethodBase method, Delegate detourDelegate)
          at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.SetShimMethod(Delegate optionalStub, Object optionalReceiver, MethodBase method)
          at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.SetShim(Delegate optionalStub, Type receiverType, Object optionalReceiver, String name, ShimBinding flags, Type returnType, Type[] parameterTypes)
          at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.SetShimPublicInstance(Delegate optionalStub, Type receiverType, Object optionalReceiver, String name, Type returnType, Type[] parameterTypes)
          at System.IO.Fakes.ShimDirectoryInfo.AllInstances.set_DeleteBoolean(Action`2 value)

    I am going to email NCover about the issue since it seems specifically related to their product, but I figured I would also post the information here for the MS team.  

    Ed,

    When you say "run through NCover" can you explain a bit what that means?

    As I just replied to Volo1234, Shims requires a special profiler that is setup by our test runner. I suspect NCover has their own profiler, so it is unlikely that ours is running.

    Regards,

    --Peter


    -- Peter Provost :: Sr. Program Manager Lead :: Visual Studio ALM

    Saturday, June 16, 2012 7:16 PM
  • I'm trying to return a test set from EF DbSet property

    public class FooContext : DbContext
    {
        public DbSet<FooEntity> FooEntities { get; set; }
    }

    DbSet constructor is protected, so I couldn't return a new instance as is but I hoping to shim AsEnumerable to test my ViewModel which uses it. (I know about IRepository pattern but I wanted to save some time shimming EF context propeties directly). 

    Thanks

    Voffka

    Monday, June 18, 2012 6:46 AM
  • Peter,

    Are there any plans on making this to where I can run all my unit tests with Fakes outside of TFS2012 and VS2012?  What I am wanting to do is run all my unit tests on the command line after I run a build from MSBuild on my local machine and make sure that I didn't break any functionality.  

    With out this, having to open up multiple VS2012 just so that I can run each projects collection of unit tests is a pain that I do not want to have to have and for a true CI should not have to do.

    Thank you,
    • Edited by Don Pavlik Wednesday, June 27, 2012 4:17 PM
    Wednesday, June 27, 2012 3:51 PM
  • Peter, 

      You set up NCover to execute your tests for you to track code coverage. The command line ends up looking similar to this "ncover xunit common.dll". 

      We're running into the same problem just doing "xunit common.dll" as part of our local and TFS 2010 build process, which I also guess is the same problem.  

      Is there a way (or going to be a way) that we can, from a command line build, have VS 2012 run our unit tests or a way that a 3rd party runner can hook into VS to not get the issue?  I'm assuming TFS 2012 build will not have this issue, but I'm months away from being able to push that into production and it doesn't solve the local command line build.  (We have about 10 solutions that are built and tested locally whenever someone does a code merge).

    Wednesday, June 27, 2012 3:57 PM
  • Peter, 

      You set up NCover to execute your tests for you to track code coverage. The command line ends up looking similar to this "ncover xunit common.dll". 

      We're running into the same problem just doing "xunit common.dll" as part of our local and TFS 2010 build process, which I also guess is the same problem.  

      Is there a way (or going to be a way) that we can, from a command line build, have VS 2012 run our unit tests or a way that a 3rd party runner can hook into VS to not get the issue?  I'm assuming TFS 2012 build will not have this issue, but I'm months away from being able to push that into production and it doesn't solve the local command line build.  (We have about 10 solutions that are built and tested locally whenever someone does a code merge).


    I believe that command actually enables a profiler, which will conflict with other profilers in use, like the one for Shims. But as I'm not 100% sure how NCover does what it does, this is only a guess.

    -- Peter Provost :: Sr. Program Manager Lead :: Visual Studio ALM

    Wednesday, September 26, 2012 2:48 PM
  • Peter,

    Are there any plans on making this to where I can run all my unit tests with Fakes outside of TFS2012 and VS2012?  What I am wanting to do is run all my unit tests on the command line after I run a build from MSBuild on my local machine and make sure that I didn't break any functionality.  

    With out this, having to open up multiple VS2012 just so that I can run each projects collection of unit tests is a pain that I do not want to have to have and for a true CI should not have to do.

    Thank you,

    Hey Don,

    You should be able to use the new command line runner to do this. vstest.console.exe is what you're looking for.

    Regards,


    -- Peter Provost :: Sr. Program Manager Lead :: Visual Studio ALM

    Wednesday, September 26, 2012 2:49 PM
  • I had the ShimNotSupportedException, and found it was caused by a unittest which loaded a class with System.Reflection.Assembly.Load(...);

    It seems the shim context could not handle the loading dynamic of assamblies files.

    Friday, September 28, 2012 10:57 AM
  • I had the ShimNotSupportedException, and found it was caused by a unittest which loaded a class with System.Reflection.Assembly.Load(...);

    It seems the shim context could not handle the loading dynamic of assamblies files.


    It is very unlikely that the ShimsContext cares. Internally it doesn't do much more in the setup code than set a private static variable. The Dispose does a bit more, but mostly just clearing out the user delegates.

    -- Peter Provost :: Sr. Program Manager Lead :: Visual Studio ALM

    Friday, September 28, 2012 9:06 PM
  • I encountered this error as well.

    In my case I accidentally referenced a fakes assembly from another test project, using resharper. Removing the fakes assembly and re-adding it the correct way fixed the problem.

    Tuesday, October 02, 2012 8:58 PM
  • Hi pgroene,

    I tried you Assembly.Load scenario and cannot repro. The following is my test code.

            [TestMethod]
            public void TestMethod1()
            {
                using (ShimsContext.Create())
                {
                    var x = Assembly.Load(@"UnitTestProject1, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null");
                }
            }
           
    Can you log a connect bug with exact repro so we can track down the issue ?

    Thanks


    Patrick Tseng MSFT - Developer - Visual Studio Team Architect

    Tuesday, October 09, 2012 5:10 PM
  • One of the possible causes is missing of the fakesconfig files which should have been generated along with your xxx.fakes.dll.

    I used a central project to generate Fake assemblies, and use other projects which depend on the central project to consume the generated Fake assemblies.

    Let's call the Fake assembly consuming project FOO.

    I did encounter the ShimNotSupportedException. After some investigation(several days!), I found that the fakesconfig files were not copied along with the Fake assemblies to the folder where FOO.dll resides. After copying fakesconfig files there, my tests passed.

    Hope this helps.

    Monday, March 04, 2013 1:53 PM
  • Some of those responses are correct.  VS2012 suggests using the runsettings file for Unit testing.  You can enable code coverage there.

    I've mainly seen this exception is while DEBUGGING.  You CANNOT debug into a shims context with code coverage enabled.  Turn CC off to debug into it.  You should be able to run the unit test with CC on without an issue.  I haven't tried with the testrunconfig CC enabled though.

    Wednesday, September 25, 2013 5:56 PM