none
.NET Framework 4.6.1 update leads to 'System.AccessViolationException at System.Text.StringBuilder.Append(System.String)' RRS feed

  • Question

  • When we install .NET Framework 4.6.1 update on server, our web application just stop working. It continiously crash with the stack trace below:

    Application: w3wp.exe
    Framework Version: v4.0.30319
    Description: The process was terminated due to an unhandled exception.
    Exception Info: System.AccessViolationException
       at System.Buffer.__Memmove(Byte*, Byte*, UInt64)
       at System.Buffer._Memmove(Byte*, Byte*, UInt64)
       at System.Buffer.Memmove(Byte*, Byte*, UInt64)
       at System.Text.StringBuilder.ThreadSafeCopy(Char*, Char[], Int32, Int32)
       at System.Text.StringBuilder.Append(Char*, Int32)
       at System.Text.StringBuilder.AppendHelper(System.String)
       at System.Text.StringBuilder.Append(System.String)
       at System.Linq.Expressions.ExpressionStringBuilder.VisitConstant(System.Linq.Expressions.ConstantExpression)
       at System.Linq.Expressions.ExpressionStringBuilder.VisitMemberAssignment(System.Linq.Expressions.MemberAssignment)
       at System.Linq.Expressions.ExpressionStringBuilder.VisitMemberInit(System.Linq.Expressions.MemberInitExpression)
       at System.Linq.Expressions.ExpressionStringBuilder.VisitMemberAssignment(System.Linq.Expressions.MemberAssignment)
       at System.Linq.Expressions.ExpressionStringBuilder.VisitMemberInit(System.Linq.Expressions.MemberInitExpression)
       at System.Linq.Expressions.ExpressionStringBuilder.VisitLambda[[System.__Canon, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]](System.Linq.Expressions.Expression`1<System.__Canon>)
       at System.Linq.Expressions.ExpressionStringBuilder.ExpressionToString(System.Linq.Expressions.Expression)

    It does work without any problems when we remove the framework update and downgrade it the previous version (.NET Framework 4.6.0). There are also other unusual errors appearing in our's application log when runnign on 4.6.1:

    System.OutOfMemoryException: Array dimensions exceeded supported range.
       at System.Text.StringBuilder.ExpandByABlock(Int32 minBlockCharCount)
       at System.Text.StringBuilder.Append(Char* value, Int32 valueCount)
       at System.Text.StringBuilder.AppendHelper(String value)
       at System.Text.StringBuilder.Append(String value)
       at System.Linq.Expressions.ExpressionStringBuilder.VisitConstant(ConstantExpression node)
       at System.Linq.Expressions.ExpressionStringBuilder.VisitMemberAssignment(MemberAssignment assignment)
       at System.Linq.Expressions.ExpressionStringBuilder.VisitMemberInit(MemberInitExpression node)
       at System.Linq.Expressions.ExpressionStringBuilder.VisitMemberAssignment(MemberAssignment assignment)
       at System.Linq.Expressions.ExpressionStringBuilder.VisitMemberInit(MemberInitExpression node)
       at System.Linq.Expressions.ExpressionStringBuilder.VisitLambda[T](Expression`1 node)
       at System.Linq.Expressions.ExpressionStringBuilder.ExpressionToString(Expression node)
    System.IndexOutOfRangeException: Index was outside the bounds of the array.
       at System.Text.StringBuilder.Append(String value)
       at System.Linq.Expressions.ExpressionStringBuilder.VisitMemberInit(MemberInitExpression node)
       at System.Linq.Expressions.ExpressionStringBuilder.VisitMemberAssignment(MemberAssignment assignment)
       at System.Linq.Expressions.ExpressionStringBuilder.VisitMemberInit(MemberInitExpression node)
       at System.Linq.Expressions.ExpressionStringBuilder.VisitLambda[T](Expression`1 node)
       at System.Linq.Expressions.ExpressionStringBuilder.ExpressionToString(Expression node)

    Any help would be appreciated

    Monday, May 14, 2018 3:18 PM

Answers

  • We have finaly found the reason. As it meaybe helpfull, I describe the reason here. 

    We had dynamic classes in our application with ToString implementation emitted. Our dynamic code did work fine untill installing these updates:

    • KB4093115
    • KB4099635
    • KB4099639
    • KB4103715
    • KB4103725
    • KB4103768

    The bellow example build for x64 target had worked untill updates has been installed and began fail after the installation:

    using System;
    using System.Reflection;
    using System.Reflection.Emit;
    using System.Text;
    using System.Threading;
    
    namespace ConsoleApplication1
    {
        class Program
        {
            public static void Main()
            {
                var myAssemblyName = new AssemblyName();
                myAssemblyName.Name = "SampleAssembly";
    
                var myAssembly = 
                    Thread.GetDomain().DefineDynamicAssembly(myAssemblyName, 
                        AssemblyBuilderAccess.RunAndSave);
    
                var myModule = 
                    myAssembly.DefineDynamicModule(myAssemblyName.Name, 
                        myAssemblyName.Name + ".dll", true);
    
                var myTypeBuilder = 
                    myModule.DefineType("Example", TypeAttributes.Public);
    
                var mb = myTypeBuilder.DefineMethod("ToString",
                    MethodAttributes.Public | MethodAttributes.ReuseSlot |
                    MethodAttributes.Virtual | MethodAttributes.HideBySig,
                    typeof(string), Type.EmptyTypes);
    
                ILGenerator gen = mb.GetILGenerator();
    
                gen.DeclareLocal(typeof(string));
                
                var stringBuilderAppend = typeof(StringBuilder).GetMethod("Append", new[] { typeof(string) });
                
                gen.Emit(OpCodes.Newobj, typeof(StringBuilder).GetConstructor(Type.EmptyTypes));
                gen.Emit(OpCodes.Stloc_0);
                
                gen.Emit(OpCodes.Ldloc_0);
                gen.Emit(OpCodes.Ldstr, "test");
                gen.Emit(OpCodes.Callvirt, stringBuilderAppend);
                gen.Emit(OpCodes.Pop);
                
                gen.Emit(OpCodes.Ldloc_0);
                gen.Emit(OpCodes.Callvirt, typeof(object).GetMethod("ToString"));
                gen.Emit(OpCodes.Stloc_0);
                gen.Emit(OpCodes.Ldloc_0);
                gen.Emit(OpCodes.Ret);
    
                var myType = myTypeBuilder.CreateType();
                myAssembly.Save(myAssemblyName.Name + ".dll");
                
                string myString = Activator.CreateInstance(myType).ToString();
    
                Console.WriteLine("Example.ToString returned: {0}", new StringBuilder().Append(myString));
                Console.ReadKey();
            }
        }
    }

    When the updates are installed the method emmited returns StringBuilder instead of string which is not expected by StringBuilder.Append method used later. It looks like instruction 

    gen.Emit(OpCodes.Stloc_0);

    doesn't copy result of internal StringBuilder.ToString call into local variable and doesn't fail for some reason. As returned value of our dynamic ToString is of string type, it's compiled and run successfully until some code doesn't use the returned value as string without type checking what is inside like StringBuilder.Append does.

    Using different variabled for internal StringBuilder instance and result fixes the problem:

    using System; using System.Reflection; using System.Reflection.Emit; using System.Text; using System.Threading; namespace ConsoleApplication1 { class Program { public static void Main() { var myAssemblyName = new AssemblyName(); myAssemblyName.Name = "SampleAssembly"; var myAssembly = Thread.GetDomain().DefineDynamicAssembly(myAssemblyName, AssemblyBuilderAccess.RunAndSave); var myModule = myAssembly.DefineDynamicModule(myAssemblyName.Name, myAssemblyName.Name + ".dll", true); var myTypeBuilder = myModule.DefineType("Example", TypeAttributes.Public); var mb = myTypeBuilder.DefineMethod("ToString", MethodAttributes.Public | MethodAttributes.ReuseSlot | MethodAttributes.Virtual | MethodAttributes.HideBySig, typeof(string), Type.EmptyTypes); ILGenerator gen = mb.GetILGenerator(); gen.DeclareLocal(typeof(StringBuilder)); var stringBuilderAppend = typeof(StringBuilder).GetMethod("Append", new[] { typeof(string) }); gen.Emit(OpCodes.Newobj, typeof(StringBuilder).GetConstructor(Type.EmptyTypes)); gen.Emit(OpCodes.Stloc_0); gen.Emit(OpCodes.Ldloc_0); gen.Emit(OpCodes.Ldstr, "test"); gen.Emit(OpCodes.Callvirt, stringBuilderAppend); gen.Emit(OpCodes.Pop); gen.Emit(OpCodes.Ldloc_0); gen.Emit(OpCodes.Callvirt, typeof(object).GetMethod("ToString")); gen.DeclareLocal(typeof(string)); gen.Emit(OpCodes.Stloc_1); gen.Emit(OpCodes.Ldloc_1); gen.Emit(OpCodes.Ret); var myType = myTypeBuilder.CreateType(); myAssembly.Save(myAssemblyName.Name + ".dll"); string myString = Activator.CreateInstance(myType).ToString(); Console.WriteLine("Example.ToString returned: {0}", new StringBuilder().Append(myString)); Console.ReadKey(); } } }

    Friday, May 18, 2018 8:46 AM

All replies

  • Did you identify specific parts of your code that reveal the problem? Maybe there are some shared (static) objects that are not used correctly by concurrent Web requests. (StringBuilder is not thread-safe).


    Monday, May 14, 2018 5:42 PM
  • Thank you for reply. I checked our code and haven't found any concurrency issues. Moreover the problem seems to be related to this https://support.microsoft.com/en-us/help/4096236/description-of-the-security-only-update-for-net-framework-4-6-4-6-1-4 particular update. The code by itself looks like the following:

    class CompiledExpressionProvider : ICompiledExpressionProvider
        {
            private readonly ConcurrentDictionary<string, Delegate> _delegates;
            public CompiledExpressionProvider()
            {
                _delegates = new ConcurrentDictionary<string, Delegate>();
            }
    
            public Delegate GetOrAdd(LambdaExpression expression, Func<LambdaExpression, Delegate> expressionCompiler)
            {
                var key = expression.ToString();
                return _delegates.GetOrAdd(key, _ => expressionCompiler(expression));
            }
        }

    The issue happening in this line

    var key = expression.ToString();
    Under the hood it uses new instance of ExpressionStringBuilder which instantiate StringBuilder for each ToString() call.  I looks like some kind of thread safety is violated inside framework with the update I've mentioned.

     
    Wednesday, May 16, 2018 6:44 AM
  • We have finaly found the reason. As it meaybe helpfull, I describe the reason here. 

    We had dynamic classes in our application with ToString implementation emitted. Our dynamic code did work fine untill installing these updates:

    • KB4093115
    • KB4099635
    • KB4099639
    • KB4103715
    • KB4103725
    • KB4103768

    The bellow example build for x64 target had worked untill updates has been installed and began fail after the installation:

    using System;
    using System.Reflection;
    using System.Reflection.Emit;
    using System.Text;
    using System.Threading;
    
    namespace ConsoleApplication1
    {
        class Program
        {
            public static void Main()
            {
                var myAssemblyName = new AssemblyName();
                myAssemblyName.Name = "SampleAssembly";
    
                var myAssembly = 
                    Thread.GetDomain().DefineDynamicAssembly(myAssemblyName, 
                        AssemblyBuilderAccess.RunAndSave);
    
                var myModule = 
                    myAssembly.DefineDynamicModule(myAssemblyName.Name, 
                        myAssemblyName.Name + ".dll", true);
    
                var myTypeBuilder = 
                    myModule.DefineType("Example", TypeAttributes.Public);
    
                var mb = myTypeBuilder.DefineMethod("ToString",
                    MethodAttributes.Public | MethodAttributes.ReuseSlot |
                    MethodAttributes.Virtual | MethodAttributes.HideBySig,
                    typeof(string), Type.EmptyTypes);
    
                ILGenerator gen = mb.GetILGenerator();
    
                gen.DeclareLocal(typeof(string));
                
                var stringBuilderAppend = typeof(StringBuilder).GetMethod("Append", new[] { typeof(string) });
                
                gen.Emit(OpCodes.Newobj, typeof(StringBuilder).GetConstructor(Type.EmptyTypes));
                gen.Emit(OpCodes.Stloc_0);
                
                gen.Emit(OpCodes.Ldloc_0);
                gen.Emit(OpCodes.Ldstr, "test");
                gen.Emit(OpCodes.Callvirt, stringBuilderAppend);
                gen.Emit(OpCodes.Pop);
                
                gen.Emit(OpCodes.Ldloc_0);
                gen.Emit(OpCodes.Callvirt, typeof(object).GetMethod("ToString"));
                gen.Emit(OpCodes.Stloc_0);
                gen.Emit(OpCodes.Ldloc_0);
                gen.Emit(OpCodes.Ret);
    
                var myType = myTypeBuilder.CreateType();
                myAssembly.Save(myAssemblyName.Name + ".dll");
                
                string myString = Activator.CreateInstance(myType).ToString();
    
                Console.WriteLine("Example.ToString returned: {0}", new StringBuilder().Append(myString));
                Console.ReadKey();
            }
        }
    }

    When the updates are installed the method emmited returns StringBuilder instead of string which is not expected by StringBuilder.Append method used later. It looks like instruction 

    gen.Emit(OpCodes.Stloc_0);

    doesn't copy result of internal StringBuilder.ToString call into local variable and doesn't fail for some reason. As returned value of our dynamic ToString is of string type, it's compiled and run successfully until some code doesn't use the returned value as string without type checking what is inside like StringBuilder.Append does.

    Using different variabled for internal StringBuilder instance and result fixes the problem:

    using System; using System.Reflection; using System.Reflection.Emit; using System.Text; using System.Threading; namespace ConsoleApplication1 { class Program { public static void Main() { var myAssemblyName = new AssemblyName(); myAssemblyName.Name = "SampleAssembly"; var myAssembly = Thread.GetDomain().DefineDynamicAssembly(myAssemblyName, AssemblyBuilderAccess.RunAndSave); var myModule = myAssembly.DefineDynamicModule(myAssemblyName.Name, myAssemblyName.Name + ".dll", true); var myTypeBuilder = myModule.DefineType("Example", TypeAttributes.Public); var mb = myTypeBuilder.DefineMethod("ToString", MethodAttributes.Public | MethodAttributes.ReuseSlot | MethodAttributes.Virtual | MethodAttributes.HideBySig, typeof(string), Type.EmptyTypes); ILGenerator gen = mb.GetILGenerator(); gen.DeclareLocal(typeof(StringBuilder)); var stringBuilderAppend = typeof(StringBuilder).GetMethod("Append", new[] { typeof(string) }); gen.Emit(OpCodes.Newobj, typeof(StringBuilder).GetConstructor(Type.EmptyTypes)); gen.Emit(OpCodes.Stloc_0); gen.Emit(OpCodes.Ldloc_0); gen.Emit(OpCodes.Ldstr, "test"); gen.Emit(OpCodes.Callvirt, stringBuilderAppend); gen.Emit(OpCodes.Pop); gen.Emit(OpCodes.Ldloc_0); gen.Emit(OpCodes.Callvirt, typeof(object).GetMethod("ToString")); gen.DeclareLocal(typeof(string)); gen.Emit(OpCodes.Stloc_1); gen.Emit(OpCodes.Ldloc_1); gen.Emit(OpCodes.Ret); var myType = myTypeBuilder.CreateType(); myAssembly.Save(myAssemblyName.Name + ".dll"); string myString = Activator.CreateInstance(myType).ToString(); Console.WriteLine("Example.ToString returned: {0}", new StringBuilder().Append(myString)); Console.ReadKey(); } } }

    Friday, May 18, 2018 8:46 AM