none
WindowsServer2016 - windowservercore runs, nanoserver hangs on run RRS feed

  • Question

  • Hi,

    new to Windows Containers and Docker and facing an issue unfortunately.

    On a WindowsServer 2016 VM (that is prepared for Containers) I can run a cmd fine on the blank windowsservercore image:

    docker run -i -t microsoft/windowsservercore:latest cmd -d

    It's starting the cmd pretty fast actually.

    Strangely I cannot do the same for nanoserver:

    docker run -it microsoft/nanoserver:latest cmd -d

    It just hangs forever without any further output in powershell after executing the command. It's only possible to kill the shell, Ctrl+C does not work.

    It shouldn't be a general prerequisite issue, because like mentioned cmd in the windowsservercore image works fine.
    It's basically same behavior when docker composing, if I base the dockerfile on nanoserver it just hangs at the first non-trivial RUN step forever. Basing the same dockerfile on windowsservercore runs through.

    Also note that I can run the cmd in nanoserver image on another Win10 dev machine without problems.
    That's docker version 17.06.1.-ce on the Win 10 machine.

    Any clues to what the issue might be are appreciated.
    The WindowsServer2016 was updated to latest docker ee version and here is Debug-ContainerHost script result as admin:

    PS C:\Windows\system32> Invoke-WebRequest https://aka.ms/Debug-ContainerHost.ps1 -UseBasicParsing | Invoke-Expression
    Checking for common problems
    Describing Windows Version and Prerequisites 
    [+] Is Windows 10 Anniversary Update or Windows Server 2016 152ms 
    [+] Has KB3192366, KB3194496, or later installed if running Windows build 14393 78ms 
    [+] Is not a build with blocking issues 84ms 
    [+] Has 'Containers' feature installed 12.02s
    Describing Docker is installed 
    [+] A Docker service is installed - 'Docker' or 'com.Docker.Service'  97ms 
    [+] Service is running 53ms 
    [+] Docker.exe is in path 2.11s 
    [-] Docker is registered in the EventLog service 372ms  
    Expected: {True}   But was:  {False}  
    71:         (Test-Path "HKLM:\SYSTEM\CurrentControlSet\Services\EventLog\Application\docker") -or (Test-Path "HKLM:\SYSTEM\CurrentControlSet\Services\EventLog\Application\DockerService") | Should Be $true   at <ScriptBlock>, <No file>: line 71Describing User has permissions to use Docker daemon 
    [+] docker.exe should not return access denied 59ms
    Describing Windows container settings are correct 
    [+] Do not have DisableVSmbOplock set to 1 62ms 
    [+] Do not have zz values set 57ms 
    [+] Do not have FDVDenyWriteAccess set to 1 55ms
    Describing The right container base images are installed 
    [+] At least one of 'microsoft/windowsservercore' or 'microsoft/nanoserver' should be installed 352ms
    Describing Container network is created 
    [+] At least one local container network is available 3.51s 
    [+] At least one NAT, Transparent, or L2Bridge Network exists 30ms 
    [+] NAT Network's vSwitch is internal 59ms 
    [+] A Windows NAT is configured if a Docker NAT network exists 403ms 
    [+] Specified Network Gateway IP for NAT network is assigned to Host vNIC 86ms 
    [+] NAT Network's internal prefix does not overlap with external IP' 45ms
    Showing output from: docker info
    Containers: 20 Running: 0 Paused: 0 Stopped: 20Images: 37
    Server Version: 17.06.1-ee-2
    Storage Driver: windowsfilter 
    Windows:
    Logging Driver: json-file
    Plugins: 
    Volume: local 
    Network: l2bridge l2tunnel nat null overlay transparent 
    Log: awslogs etwlogs fluentd json-file logentries splunk syslog
    Swarm: inactive
    Default Isolation: process
    Kernel Version: 10.0 14393 (14393.1593.amd64fre.rs1_release.170731-1934)
    Operating System: Windows Server 2016
    StandardOSType: windows
    Architecture: x86_64
    CPUs: 4
    Total Memory: 16GiB
    Name: xxx
    ID: PQ7L:FTH4:G7IN:7BBM:YMM4:S2SZ:LCQ2:3QI3:UJBL:KY53:VY6V:4ABG

    Docker Root Dir: D:\docker\data
    Debug Mode (client): false
    Debug Mode (server): false
    Registry: https://index.docker.io/v1/
    Experimental: false
    Insecure Registries: 127.0.0.0/8
    Live Restore Enabled: false
    Showing output from: docker version
    Client: Version:      17.06.1-ee-2 
    API version:  1.30 
    Go version:   go1.8.3 
    Git commit:   8e43158 
    Built:        Wed Aug 23 21:16:53 2017 
    OS/Arch:      windows/amd64
    Server: Version:      17.06.1-ee-2 
    API version:  1.30 (minimum version 1.24) 
    Go version:   go1.8.3 
    Git commit:   8e43158 
    Built:        Wed Aug 23 21:25:53 2017 
    OS/Arch:      windows/amd64 
    Experimental: false
    Showing output from: docker network ls
    NETWORK ID          NAME                DRIVER              SCOPE
    569da92bb583        nat                 nat                 local
    c6edfee623d6        none                null                local
    Getting Warnings & errors in the Windows event logs from the last 24 hours
    Logs saved to C:\Windows\system32\logs_20170828-191509.csv
    Getting Docker for Windows daemon logs from the last execution   
    Note: More logs are available at C:\Users\dmm\AppData\Local\Docker. Only showing the latest.
    Get-Content : Cannot find path 'C:\Users\dmm\AppData\Local\Docker\log.txt' because it does not exist.At line:291 char:1+ Get-Content "$($ENV:LOCALAPPDATA)\Docker\log.txt" | Select-String "Wi ...+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~    + CategoryInfo          : ObjectNotFound: (C:\Users\dmm\Ap...\Docker\log.txt:String) [Get-Content], ItemNotFoundEx   ception    + FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand





    Monday, August 28, 2017 5:59 PM

Answers

  • I can confirm now that the Bitdefender Endpoint Security prevents nanoserver from running.

    IT turned it off for a while and everything was well. Turned it on again and back to freezing nanoservers ...

    Anyway, thanks very much for your input on this. Maybe it helps someone else another time.

    Cheers!

    • Marked as answer by bagofpecans Friday, September 1, 2017 3:29 PM
    Friday, September 1, 2017 3:29 PM

All replies

  • Weird. Can you test your docker command in a clean Command Prompt (cmd), outside of PowerShell? Are you running this on a remote machine or directly on the container host?
    Wednesday, August 30, 2017 3:32 AM
  • Hi,

    thanks for responding. Yes it is weird, I totally agree. Was never expecting something fundamental like that to not just work :(

    Tried the same thing on a second identically prepared corporate VM and the result is exactly the same. Anytime the  nanoserver image is involved instead of the windowsservercore one the result is a frozen shell which I have to kill. Anything with windowsservercore is perfectly fine. Frustrating.

    I was running the commands in powershell on the container host directly since it was kind of first stage testing.
    As suggested I now tried running nano and servercore in CMD (which I didn't previously), results are unfortunately the same.

    By now the only thing I can think of is the Bitdefender Endpoint Security that is running on the corporate VMs. Which would still dazzle me since the windowsservercore just runs without issues. Unfortunately having that turned off, even temporarily, is kinda problematic due to company guidelines. Will have to see if it is possible and report.

    Unless someone can confirm that the described setup with Bitdefender Security works (or doesn't) or our IT department turns it off for me I am out of ideas...

    Thanks again for taking the time :)

    Thursday, August 31, 2017 8:28 AM
  • Can you verify KB3192366 and KB3194496 are installed on the host?
    Thursday, August 31, 2017 9:04 AM
  • I have installed:

    [01]: KB3199986
    [02]: KB4023834
    [03]: KB4035631
    [04]: KB4034658

    They are not explicitly installed. It looks as if KB4034658 includes KB3194496 which in turn includes KB3192366? I am no expert at Windows Hotfixes though.

    The Debug-ContainerHost script (if correct) states:

    [+] Has KB3192366, KB3194496, or later installed if running Windows build 14393 78ms 

    Thursday, August 31, 2017 12:15 PM
  • Some more info, I ran docker -D to get more logging:

    time="2017-08-31T18:18:02.797157300+02:00" level=debug msg="HCSShim::CreateContainer id=e9d239d8f9933c43160e027e7037b0148b070123
    c09e4b50a3afcd3a7fa5f828 config={\"SystemType\":\"Container\",\"Name\":\"e9d239d8f9933c43160e027e7037b0148b070123c09e4b50a3afcd3
    a7fa5f828\",\"Owner\":\"docker\",\"IsDummy\":false,\"VolumePath\":\"\\\\\\\\?\\\\Volume{7a298151-8ca9-11e7-9672-c70683b83c73}\",
    \"IgnoreFlushesDuringBoot\":true,\"LayerFolderPath\":\"C:\\\\ProgramData\\\\docker\\\\windowsfilter\\\\e9d239d8f9933c43160e027e7
    037b0148b070123c09e4b50a3afcd3a7fa5f828\",\"Layers\":[{\"ID\":\"a96c80a5-b159-54b2-961b-dc2ae2738eb2\",\"Path\":\"C:\\\\ProgramD
    ata\\\\docker\\\\windowsfilter\\\\9942fbad5b63f5bacc5d28636c3f94fd42b1f44e78d48e5a410335fe2717dc04\"},{\"ID\":\"d5213f4f-abe9-58
    72-8ba3-7d2f1ad6b90c\",\"Path\":\"C:\\\\ProgramData\\\\docker\\\\windowsfilter\\\\ecb78e39ecc12041cba216af2da80a6baf400d6a0239d3
    0afca3151460c7873b\"}],\"HostName\":\"e9d239d8f993\",\"MappedDirectories\":[],\"HvPartition\":false,\"EndpointList\":[\"ceecc449
    -0c34-4c1c-980c-c1a2257ee5bc\"],\"Servicing\":false,\"AllowUnqualifiedDNSQuery\":true}"
    time="2017-08-31T18:18:03.383189200+02:00" level=debug msg="HCSShim::CreateContainer succeeded id=e9d239d8f9933c43160e027e7037b0
    148b070123c09e4b50a3afcd3a7fa5f828 handle=39400864"
    time="2017-08-31T18:18:03.387187900+02:00" level=debug msg="libcontainerd: Create() id=e9d239d8f9933c43160e027e7037b0148b070123c
    09e4b50a3afcd3a7fa5f828, Calling start()"
    time="2017-08-31T18:18:03.389188700+02:00" level=debug msg="libcontainerd: starting container  e9d239d8f9933c43160e027e7037b0148
    b070123c09e4b50a3afcd3a7fa5f828"
    time="2017-08-31T18:18:03.390188700+02:00" level=debug msg="HCSShim::Container::Start id=e9d239d8f9933c43160e027e7037b0148b07012
    3c09e4b50a3afcd3a7fa5f828"

    That is where it stops exactly, at Container::Start

    Nothing happening after that



    Thursday, August 31, 2017 4:21 PM
  • I can confirm now that the Bitdefender Endpoint Security prevents nanoserver from running.

    IT turned it off for a while and everything was well. Turned it on again and back to freezing nanoservers ...

    Anyway, thanks very much for your input on this. Maybe it helps someone else another time.

    Cheers!

    • Marked as answer by bagofpecans Friday, September 1, 2017 3:29 PM
    Friday, September 1, 2017 3:29 PM
  • Thanks for the follow up, be sure to hit us up if you run into any other issues.
    Friday, September 1, 2017 5:17 PM