none
DNS Resolution Failure only on Windows Server 2012 R2 RRS feed

  • Question

  • I've encountered a very bizarre bug that I've narrowed down to a specific scenario but due to potential impact I need to try and find out why. 

    Application A v 1.0 built for `.NET 4.7.2` runs fine on `Windows Server 2012 R2`
    Application A v 2.0 also built for `.NET 4.7.2` but now using a `.Net Standard 2.0` shared library suddenly causes the tcp stack to fail (specifically all DNS resolution fails, even internal destinations) 

    This bug only occurs on `2012 R2`. `2016` works just fine. 
    This bug occurs even if only the assembly for the shared library is included, no actual instances declared or references made, not even a using statement, just having the shared library in the dependencies causes this. 

    Outside of the application DNS resolution works fine. 
    FusionLogs are functionally identical between what gets loaded on my `Windows 10 machine` and what gets loaded on `2012 R2`. Only difference being the encoded folder names for the native images loaded.

    Here's a list of all relevant dependencies from the shared library

    BouncyCastle.Crypto v 1.8.5: 
        Org.BouncyCastle.Crypto.Parameters;    
        Org.BouncyCastle.OpenSsl;    
        Org.BouncyCastle.Security;
    Common.Logging v 3.4.1:
    DnsClient v 1.2.0:
    Microsoft.CSharp v 4.7.0:
    Mimekit v 2.4.1:
    NewtonSoft.Json v 12.0.1:
    System v 4.0.0.0:    
        System.Collections.Generic;    
        System.Collections.Concurrent;    
        System.IO;    
        System.IO.Compression.FileSystem;    
        System.Drawing;
        System.Numerics;
        System.Runtime.Serialization;
        System.Net;
        System.Net.Http;
        System.Net.Http.Headers;
        System.Security.Cryptography;
        System.Text;
        System.Threading;
        System.Threading.Tasks;
        System.Xml;

    Exception:

    17:16:21 [1] [Info] - DNS Query against 'google.com'
    Unhandled Exception: System.Net.Sockets.SocketException: No such host is known   
       at System.Net.Dns.GetAddrInfo(String name)   
       at System.Net.Dns.InternalGetHostByName(String hostName, Boolean includeIPv6)   
       at System.Net.Dns.GetHostEntry(String hostNameOrAddress)
       at DNSAPITestApp.Tester..ctor() in ...
       at DNSAPITestApp.Program.Main(String[] args) in ...

    I've searched extensively online however there's just too much white noise of similar DNS issues that still having nothing to do with this specific scenario or why this could be happening (and just as importantly how to confirm any theories).

    Any help would be greatly appreciated. I can't post much of the code but if there are any specific areas where a particular dependency is used that anyone would like to see I can add snippets. 

    EDIT: Adding example test code. 
    EDIT2: Updating example test code.

    using NLog;
    using NLog.Config;
    using NLog.Targets;
    using System.Net;
    using System;
    using System.Reflection;
    
    namespace DNSAPITestApp
    {
        public class Tester
        {
            private readonly Logger _logger;
    
            public Tester()
            {
                var logConfig = new LoggingConfiguration();
    
                var consoleTarget = new ConsoleTarget("consoleLog")
                {
                    Layout = @"${date:format=HH\:mm\:ss} [${threadid}] [${level}] - ${message} ${exception}",
                    OptimizeBufferReuse = true
                };
                
                logConfig.AddTarget(consoleTarget);
                
                logConfig.AddRuleForAllLevels(consoleTarget);
                LogManager.Configuration = logConfig;
                _logger = LogManager.GetCurrentClassLogger();
            }
    
            public void Run()
            {
                TypeInfo t = typeof(Dns).GetTypeInfo();
    
                _logger.Info($"System.Net.Dns Assembly Location: {t.Assembly.Location}");
    
                _logger.Info($"System.Net.Dns Module: {t.Module.ToString()}");
    
                _logger.Info($"System.Net.Dns Underlying System type: {t.UnderlyingSystemType.FullName}\nUnderlying Assembly Location: {t.UnderlyingSystemType.Assembly.Location}");
    
                var result = Dns.GetHostEntry("www.google.com");
    
                foreach (var address in result.AddressList)
                    _logger.Info($"Address: {address.ToString()}");
    
                Console.ReadLine();
            }
        }
    }
    

    Thursday, January 23, 2020 2:40 PM

Answers

  • The library name is the issue. When you compile that library it is DnsApi.dll. That happens to be the same name as the Windows DLL that contains the DNS native logic. When it tries to load the system DLL it will find yours, because it searches the app directory first and then tries to use it but, of course, it doesn't work.

    Rename the class library to something else and try again.


    Michael Taylor http://www.michaeltaylorp3.net

    • Marked as answer by Sylvadune Tuesday, January 28, 2020 7:17 PM
    Tuesday, January 28, 2020 6:58 PM
    Moderator

All replies

  • Hello Sylvadune,

    This forum is for "Discuss general issues about developing applications for Windows." It is C++, Win32 API focused.

    Since this issue is C# related I'll move it to C# forum for more professionally support.

    Best regards,

    Rita


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Friday, January 24, 2020 2:45 AM
  • Please post your test code. Also clarify what some shared library has to do with this because the stack trace indicates you are calling the constructor for some `Tester` class and it is calling `Dns.GetHostEntry`. Given that the `Tester` class is inside your main app (which I assume is a .NET 4.7.2 console app) then a shared library isn't relevant at this point.

    Michael Taylor http://www.michaeltaylorp3.net

    Friday, January 24, 2020 3:01 PM
    Moderator
  • Please post your test code. Also clarify what some shared library has to do with this because the stack trace indicates you are calling the constructor for some `Tester` class and it is calling `Dns.GetHostEntry`. Given that the `Tester` class is inside your main app (which I assume is a .NET 4.7.2 console app) then a shared library isn't relevant at this point.

    Michael Taylor http://www.michaeltaylorp3.net

    I added the requested test code. However, this code only fails if the shared library is included as a dependency. That's why the shared library matters as regardless of ever instantiating anything from the shared library, if it's included as a dependency the test code fails. If it is removed the test code works fine. 
    Monday, January 27, 2020 5:16 PM
  • Running theory. That shared library, being based on .NET Standard, is pulling in the NuGet package System.Net.NameResolution which is where `Dns` is defined. However without that library reference then you have just a .NET Framework app which is using the older 'Dns' class from the core framework.

    You need to look at the dependencies that the shared library is using. It should be referencing the core framework when used against a NF app and the NuGet package when used against a .NET Core (NC) app. If it is trying to use the NuGet package for both then you're going to run into issues. You might try getting the type info from `Dns` to see where it is coming from.

    I took your code snippet and dropped it into a NF console app. Then I added a separate .NET Standard class library and added that as a reference in the console app. Your code worked correctly for me. The only thing I see different is that your output shows an info  message about DNS Query but that isn't in your main code anywhere. Either the code you posted didn't generate that output or something is injecting itself between you and `Dns`.

    Another thing to look at is the actual version of .NET installed on your servers. Make sure the installed frameworks are the same. When dealing with interop between NF and NC apps the target framework becomes very important. For older NF versions there was a lot of thunk assemblies generated into the output which can cause issues. Newer NF versions don't have that issue.

    Finally I notice you have Castle loaded. I'm assuming that this isn't actually being used in your simplified test case but proxy-type libraries can have issues when you mix NF/NC code.


    Michael Taylor http://www.michaeltaylorp3.net

    Monday, January 27, 2020 5:46 PM
    Moderator
  • Wonderful idea! I'm working on testing some of this now. 

    You're right on the disconnect with the test code I posted and the output I originally posted. I'm on a different computer at the moment and don't have access to the QnD test app I originally made for this so I just replicated the same general steps it went through. 

    The dependencies I listed are from the shared library. I'm worried that it might be because of weirdness from Castle but wanted to chase down other possibilities first before going down that rabbit hole. The shared library is using it for OpenSSL tasks (creating and reading RSA keys and signing https requests) 

    thank you so much for your help! I'll follow up here once I have the results of those tests. 

    Monday, January 27, 2020 6:15 PM
  • Updated test app on Windows 10 dev machine: 

    14:30:59 [1] [Info] - System.Net.Dns Assembly Location: C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll
    14:30:59 [1] [Info] - System.Net.Dns Module: System.dll
    14:30:59 [1] [Info] - System.Net.Dns Underlying System type: System.Net.Dns
    Underlying Assembly Location: C:\WINDOWS\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll
    14:30:59 [1] [Info] - Address: 172.217.9.196

    Windows Server 2012 R2 (No other DNS issues, resolution issue is exclusive to this app):

    14:24:12 [1] [Info] - System.Net.Dns Assembly Location: C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll
    14:24:12 [1] [Info] - System.Net.Dns Module: System.dll
    14:24:12 [1] [Info] - System.Net.Dns Underlying System type: System.Net.Dns
    Underlying Assembly Location: C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll
    
    Unhandled Exception: System.Net.Sockets.SocketException: No such host is known
       at System.Net.Dns.GetAddrInfo(String name)
       at System.Net.Dns.InternalGetHostByName(String hostName, Boolean includeIPv6)
       at System.Net.Dns.GetHostEntry(String hostNameOrAddress)
       at DNSAPITestApp.Tester.Run() in ... \Tester.cs:line 41
       at DNSAPITestApp.Program.Main(String[] args) in ... Program.cs:line 15

    The Dns Class is only used in the test app. The shared library does not directly make use of it. 
    This has given me an avenue for more tests related to System.Net.Sockets or other .dll's that the shared library is referencing see if anything comes up different. 

    Will update here, if you have any other ideas or a follow up on maybe I'm performing this test wrong just let me know :) 


    • Edited by Sylvadune Tuesday, January 28, 2020 2:31 PM
    Monday, January 27, 2020 7:37 PM
  • I'm starting to wonder if the issue is with the server configuration and not your code. Does this happen on all servers or just a particular one? Can that server ping the DNS name from a command line correctly? Is the server configured to allow IPv4 and v6?

    Next option, turn on network tracing in the framework using this. See what gets printed out.


    Michael Taylor http://www.michaeltaylorp3.net

    Monday, January 27, 2020 8:07 PM
    Moderator
  • I'm starting to wonder if the issue is with the server configuration and not your code. Does this happen on all servers or just a particular one? Can that server ping the DNS name from a command line correctly? Is the server configured to allow IPv4 and v6?

    Next option, turn on network tracing in the framework using this. See what gets printed out.


    Michael Taylor http://www.michaeltaylorp3.net

    That was my first thought as well. 
    However, dns queries from the server work fine (command line or otherwise). 

    This issue is repeatable on VMWare instances, AWS Windows 2012 R2 AMI, and on Windows 7. 
    Tested with fresh installs of each confirming the correct NF runtime is installed. 

    Monday, January 27, 2020 8:25 PM
  • network trace log on Windows 10 Dev Machine:

    System.Net.Sockets Verbose: 0 : [19716] Entering DNS::GetHostEntry(www.google.com)
        ProcessId=24884
        DateTime=2020-01-27T20:38:55.2067253Z
    System.Net.Sockets Verbose: 0 : [19716] Entering DNS::GetHostByName(www.google.com)
        ProcessId=24884
        DateTime=2020-01-27T20:38:55.2107274Z
    System.Net.Sockets Verbose: 0 : [19716] Exiting DNS::GetHostByName() 	-> IPHostEntry#64844482
        ProcessId=24884
        DateTime=2020-01-27T20:38:55.2167240Z
    System.Net.Sockets Verbose: 0 : [19716] Exiting DNS::GetHostEntry() 	-> IPHostEntry#64844482
        ProcessId=24884
        DateTime=2020-01-27T20:38:55.2167240Z

    On Windows Server 2012 R2

    System.Net.Sockets Verbose: 0 : [2972] Entering DNS::GetHostEntry(www.google.com)
        ProcessId=2968
        DateTime=2020-01-27T20:38:13.6613170Z
    System.Net.Sockets Verbose: 0 : [2972] Entering DNS::GetHostByName(www.google.com)
        ProcessId=2968
        DateTime=2020-01-27T20:38:13.6613170Z
    System.Net Error: 0 : [2972] Exception in AppDomain#64844482::UnhandledExceptionHandler - No such host is known.
       at System.Net.Dns.GetAddrInfo(String name)
       at System.Net.Dns.InternalGetHostByName(String hostName, Boolean includeIPv6)
       at System.Net.Dns.GetHostEntry(String hostNameOrAddress)
       at DNSAPITestApp.Tester.Run() in ... \Tester.cs:line 49
       at DNSAPITestApp.Program.Main(String[] args) in ... \Program.cs:line 15
        ProcessId=2968
        DateTime=2020-01-27T20:38:13.6769453Z

    Monday, January 27, 2020 8:43 PM
  • I took your sample code and dropped it onto a clean 2012 R2 server with .NET 4.7.2 installed and the code behaved correctly.

    Michael Taylor http://www.michaeltaylorp3.net

    Monday, January 27, 2020 11:02 PM
    Moderator
  • I took your sample code and dropped it onto a clean 2012 R2 server with .NET 4.7.2 installed and the code behaved correctly.

    Michael Taylor http://www.michaeltaylorp3.net

    That would be expected as you do not have the shared library that is causing this issue. 

    The frustrating bit is that i cannot share this shared library so that you can reproduce the issue. But I can give some information about it, like the complete dependency list in the original post. 

    I was hoping someone might have some insight into anything with those dependencies that behaves differently under the hood between Win 2012 architecture and Win 2016

    Tuesday, January 28, 2020 2:23 PM
  • Given the callstack I cannot imagine why the shared library would be causing an issue. At this point it shouldn't have any even been loaded because .NET does lazy loading. If it didn't load then none of its dependencies would either.

    What makes you believe the shared library is the cause? Is it because when you remove that dependency the code starts working? What dependencies does it have? 

    When you bring up a new 2012 R2 machine you said the problem replicates itself but I assume you first had to install .NET 4.7.2 on the server which would have triggered the install of a KB article as well. You did install them correct?


    Michael Taylor http://www.michaeltaylorp3.net

    Tuesday, January 28, 2020 2:46 PM
    Moderator
  • The moment I remove the shared library's .dll from the same directory the test app is in, it works. (removing the handful of additional .dll's the library brings had no affect, only removing the library's specific .dll resolved the issue.) 

    And yes, on my local VBox I had to install it, and on our VMWare hypervisor I had to install 472. 
    AWS AMI had it pre-installed. 

    As a side note/reminder, this issue is also replicated on Windows 7. 

    • Edited by Sylvadune Tuesday, January 28, 2020 4:22 PM
    Tuesday, January 28, 2020 4:13 PM
  • What is the name of the shared DLL? It is being magically loaded by Castle or something such that it is running any sort of code that initializes things that could get in the way? Maybe take a look at the loaded modules in Process Explorer or equivalent to see what it looks like, the app domains and assemblies loaded, etc.

    Michael Taylor http://www.michaeltaylorp3.net

    Tuesday, January 28, 2020 5:00 PM
    Moderator
  • What is the name of the shared DLL? It is being magically loaded by Castle or something such that it is running any sort of code that initializes things that could get in the way? Maybe take a look at the loaded modules in Process Explorer or equivalent to see what it looks like, the app domains and assemblies loaded, etc.

    Michael Taylor http://www.michaeltaylorp3.net

    The shared library is internally developed. The name is DNSAPI. 
    I don't have any compiler flags or assembly commands in said library. There are some static class constructors but that shouldn't matter methinks. 

    Investigating now with Process Explorer. Will update here. 
    Tuesday, January 28, 2020 5:12 PM
  • Here's the .NET Assemblies loaded and referenced on the Win 2012 R2 server

    

    And here is the same info on my Windows 10 Dev machine:

    I don't really see a functional difference. 

    However, on the Disk and Network tab I noticed that on my Win 10 machine under Network I/O Other there was 3 (no delta no bytes) 
    And 0 for anything on the server. 
    Same tab under Disk I/O Windows 10 machine, 16 kb read, 36 kb written. Server 0. 

    Tuesday, January 28, 2020 6:00 PM
  • The library name is the issue. When you compile that library it is DnsApi.dll. That happens to be the same name as the Windows DLL that contains the DNS native logic. When it tries to load the system DLL it will find yours, because it searches the app directory first and then tries to use it but, of course, it doesn't work.

    Rename the class library to something else and try again.


    Michael Taylor http://www.michaeltaylorp3.net

    • Marked as answer by Sylvadune Tuesday, January 28, 2020 7:17 PM
    Tuesday, January 28, 2020 6:58 PM
    Moderator
  • The library name is the issue. When you compile that library it is DnsApi.dll. That happens to be the same name as the Windows DLL that contains the DNS native logic. When it tries to load the system DLL it will find yours, because it searches the app directory first and then tries to use it but, of course, it doesn't work.

    Rename the class library to something else and try again.


    Michael Taylor http://www.michaeltaylorp3.net

    That was it! Thank you so much!!! We've been chasing this for over a month. 
    Tuesday, January 28, 2020 7:17 PM