Answered by:
Bluescreen and applicationHost.config overwritten with zeroes

Question
-
User515338469 posted
I have an application which makes some changes to applicationHost.config. Every time I run this application, my system crashes with a bluescreen, and applicationHost.config is overwritten with zeroes.
I have been able to recover a recent valid applicationHost.config file from c:\inetpub\history and reproduce the problem every time I run my application.
This started happening after a Windows update that was installed yesterday (2019-10-10).
What exactly my application does is quite complicated. For example, it creates applications in IIS by running appcmd. Some modifications to the config are done by a .NET application, which uses Microsoft.Web.Administration. I also cannot tell at which exact point the crash happens, but I am working on getting more details.
System information:
Windows 10 1903 (it is a development machine)
Is this a known issue with the current Windows / IIS version?
What more information can I provide to get help, and what debugging steps would you recommend for me to do?
Friday, October 11, 2019 11:26 AM
Answers
-
User-460007017 posted
Hi fw-flw,
cldflt.sys belong to cloud file mini filter driver. Since the error message is thrown from native code. Could you fix this issue by rolling back these updates? I believe this issue is a compatibility issue.
I'm afraid you could only accept rolling back update as a workaround.
If you need to fix this, You may need to support a ticket to Microsoft since PG need to collect business impact.
If the reply is helpful, it is appreciated if you could mark the reply as answer.
Best Regards,
Jokies Ding
- Marked as answer by Anonymous Tuesday, September 28, 2021 12:00 AM
Monday, October 14, 2019 9:08 AM
All replies
-
User515338469 posted
More research shows that my application also runs DISM to install IIS and IIS features, and that (according to logs) this may have been the last operation before the crash.
The DISM command-line is as follows:
dism /online /norestart /enable-feature /featurename:IIS-ApplicationDevelopment /featurename:IIS-ASPNET /featurename:IIS-ASPNET45 /featurename:IIS-CGI /featurename:IIS-CommonHttpFeatures /featurename:IIS-HealthAndDiagnostics /featurename:IIS-HttpCompressionDynamic /featurename:IIS-HttpErrors /featurename:IIS-HttpLogging /featurename:IIS-HttpRedirect /featurename:IIS-HttpTracing /featurename:IIS-IPSecurity /featurename:IIS-ISAPIExtensions /featurename:IIS-ISAPIFilter /featurename:IIS-LoggingLibraries /featurename:IIS-ManagementScriptingTools /featurename:IIS-NetFxExtensibility /featurename:IIS-NetFxExtensibility45 /featurename:IIS-Performance /featurename:IIS-RequestMonitor /featurename:IIS-RequestFiltering /featurename:IIS-Security /featurename:IIS-URLAuthorization /featurename:IIS-WebServer /featurename:IIS-WebServerManagementTools /featurename:IIS-WebServerRole /all
However, by running this command from the console, I cannot reproduce the crash.
Friday, October 11, 2019 11:54 AM -
User515338469 posted
DISM is innocent.
The crash is caused by a .NET app using Microsoft.Web.Administration to set some ISAPI and CGI Restriction rules. Given the current configuration, this app would not make any actual changes. Removing and adding the rules manually in IIS Manager does not cause a crash. Other operations using the same .NET app and Microsoft.Web.Administration do not cause crashes.
Friday, October 11, 2019 12:53 PM -
User515338469 posted
More shit has come to light:
The BSOD is caused by cldflt.sys, which according to various search results (often also related to BSOD situations) is a part of or used by Microsoft OneDrive.
The crash is caused by the second CommitChanges() call in this sequence:
- 1. Remove ISAPI/CGI restriction
- 2. CommitChanges()
- 3. Add ISAPI/CGI restriction
- 4. CommitChanges()
Another fun fact: When I ran this .NET application in the debugger in Visual Studio (which also reproduced the crash), the user settings file of the project was also overwritten with zeroes after the crash, causing Visual Studio to be unable to load the project until I deleted the .user file.
Friday, October 11, 2019 1:45 PM -
User690216013 posted
Where did you save your source code? It is strongly recommended that you don't save your code in OneDrive mapped folders as most of Microsoft development tools (IIS/Visual Studio) might not be tested to support such setup.
If you really want to back up your code periodically, learn to use a source code control system like Git.
Friday, October 11, 2019 4:54 PM -
User515338469 posted
Neither my source code, nor my VS project files, nor anything related to IIS or my application is stored in OneDrive.
This is why I don't understand why an API call to change the IIS configuration would cause a OneDrive driver to BSOD.Friday, October 11, 2019 7:23 PM -
User-460007017 posted
Hi fw-flw,
Could you fix this issue by removing the update you installed in 10/10/2019? If this issue can be fixed by removing the update, We would know that some change applied in this update cause the crash.
It is recommended to restore the update as a workaround.
If you need to figure out the root cause.
1.Please find the dump file generated when the server become BSOD. The location would be C:\Winows\memory.dmp
2.We need to use dump analysis tool like WINDbg or Debug diagnostic tool to analyze the dump file
If the exception come from managed code, we need to figure out this issue come from native code or managed code. It will help us find the root cause.
If you are not expert in dump analysis, it is recommended to open a support ticket to https://support.microsoft.com/en-us.
Professional support engineer will help you handle this. If it is proved to be bug, they will help you report this issue and ask for a workaround or solution.
If the reply is helpful, it is appreciated if you could mark it as answer.
Best Regards,
Jokies Ding
Monday, October 14, 2019 3:42 AM -
User-2064283741 posted
I'm not sure this is an IIS issue.
Does changing the app host config by hand / appcmd result in this behavior?
Monday, October 14, 2019 6:29 AM -
User515338469 posted
Hi,
Changing the app host config by hand (using "IIS Manager" - the Microsoft one, not our app, which has the same name - see below) does not cause a crash.
Here is the output of WinDBG. I can also provide the complete memory dump, if it is still needed
(IISManager.exe is our app, which is a .NET 4.6.1 app that uses Microsoft.Web.Administration)
Also note that (as mentioned earlier), our IISManager.exe makes several changes to app host config BEFORE the change to ISAPI/CGI restrictions that do NOT cause a crash.
Microsoft (R) Windows Debugger Version 6.3.9600.17336 AMD64 Copyright (c) Microsoft Corporation. All rights reserved. Loading Dump File [C:\Lab\MEMORY.DMP] Kernel Bitmap Dump File: Only kernel address space is available ************* Symbol Path validation summary ************** Response Time (ms) Location Deferred https://msdl.microsoft.com/download/symbols Symbol search path is: https://msdl.microsoft.com/download/symbols Executable search path is: Windows 8 Kernel Version 18362 MP (12 procs) Free x64 Product: WinNt, suite: TerminalServer SingleUserTS Built by: 18362.1.amd64fre.19h1_release.190318-1202 Machine Name: Kernel base = 0xfffff801`7dc00000 PsLoadedModuleList = 0xfffff801`7e048210 Debug session time: Fri Oct 11 16:06:52.909 2019 (UTC + 2:00) System Uptime: 0 days 0:35:56.657 Loading Kernel Symbols ......................................Page 20106bd67 too large to be in the dump file. ......................... ................................................................ ................................................................ ........................................................ Loading User Symbols PEB is paged out (Peb.Ldr = 000000e7`aa57c018). Type ".hh dbgerr001" for details Loading unloaded module list ................................ ******************************************************************************* * * * Bugcheck Analysis * * * ******************************************************************************* Use !analyze -v to get detailed debugging information. BugCheck 3B, {c0000005, fffff8017afcfbfe, ffffab8c3d1dd800, 0} Probably caused by : cldflt.sys ( cldflt!HsmiFltPostECPCREATE+1da ) Followup: MachineOwner --------- 2: kd> !analyze -v ******************************************************************************* * * * Bugcheck Analysis * * * ******************************************************************************* SYSTEM_SERVICE_EXCEPTION (3b) An exception happened while executing a system service routine. Arguments: Arg1: 00000000c0000005, Exception code that caused the bugcheck Arg2: fffff8017afcfbfe, Address of the instruction which caused the bugcheck Arg3: ffffab8c3d1dd800, Address of the context record for the exception that caused the bugcheck Arg4: 0000000000000000, zero. Debugging Details: ------------------ EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - The instruction at 0x%p referenced memory at 0x%p. The memory could not be %s. FAULTING_IP: cldflt!HsmiFltPostECPCREATE+1da fffff801`7afcfbfe f60201 test byte ptr [rdx],1 CONTEXT: ffffab8c3d1dd800 -- (.cxr 0xffffab8c3d1dd800;r) rax=ffffc18df636fae8 rbx=0000000000000000 rcx=0000000000000000 rdx=0000000000000014 rsi=ffffab8c3d1ded60 rdi=ffffc18dde39b720 rip=fffff8017afcfbfe rsp=ffffab8c3d1de1f0 rbp=ffffab8c3d1de240 r8=0000000000000000 r9=7fffc18dde39b7a0 r10=fffff8017dc663d0 r11=ffffab8c3d1de1e0 r12=0000000000000014 r13=0000000000000000 r14=0000000000000000 r15=fffff8017af8d000 iopl=0 nv up ei pl zr na po nc cs=0010 ss=0018 ds=002b es=002b fs=0053 gs=002b efl=00050246 cldflt!HsmiFltPostECPCREATE+0x1da: fffff801`7afcfbfe f60201 test byte ptr [rdx],1 ds:002b:00000000`00000014=?? Last set context: rax=ffffc18df636fae8 rbx=0000000000000000 rcx=0000000000000000 rdx=0000000000000014 rsi=ffffab8c3d1ded60 rdi=ffffc18dde39b720 rip=fffff8017afcfbfe rsp=ffffab8c3d1de1f0 rbp=ffffab8c3d1de240 r8=0000000000000000 r9=7fffc18dde39b7a0 r10=fffff8017dc663d0 r11=ffffab8c3d1de1e0 r12=0000000000000014 r13=0000000000000000 r14=0000000000000000 r15=fffff8017af8d000 iopl=0 nv up ei pl zr na po nc cs=0010 ss=0018 ds=002b es=002b fs=0053 gs=002b efl=00050246 cldflt!HsmiFltPostECPCREATE+0x1da: fffff801`7afcfbfe f60201 test byte ptr [rdx],1 ds:002b:00000000`00000014=?? Resetting default scope DEFAULT_BUCKET_ID: WIN8_DRIVER_FAULT BUGCHECK_STR: 0x3B PROCESS_NAME: IISManager.exe CURRENT_IRQL: 0 ANALYSIS_VERSION: 6.3.9600.17336 (debuggers(dbg).150226-1500) amd64fre LAST_CONTROL_TRANSFER: from fffff8017afd0129 to fffff8017afcfbfe STACK_TEXT: ffffab8c`3d1de1f0 fffff801`7afd0129 : ffffc18d`f636fae8 ffffab8c`3d1de360 00000000`00000000 00000000`00000000 : cldflt!HsmiFltPostECPCREATE+0x1da ffffab8c`3d1de280 fffff801`81d03c03 : ffffc18d`f636fae8 ffffab8c`3d1de360 ffffc18d`f636fa00 00000000`00016bf0 : cldflt!HsmFltPostQUERY_OPEN+0x29 ffffab8c`3d1de310 fffff801`81d0243c : 00000000`00000000 ffffc18d`dae1dd00 ffffc18d`f4c87268 00000000`00000000 : FLTMGR!FltpPerformPostCallbacks+0x3e3 ffffab8c`3d1de3e0 fffff801`7dc89aac : ffffab8c`3d1de480 ffffab8c`3d1ded0c ffffc18d`dac1e8f0 ffffc18d`ee3e6010 : FLTMGR!FltpPostFsFilterOperation+0x2c ffffab8c`3d1de410 fffff801`7e45010d : 00000000`00000000 ffffc18d`dae1ddc0 ffffab8c`3d1de540 fffff801`829ddda0 : nt!FsFilterPerformCompletionCallbacks+0x4c ffffab8c`3d1de440 fffff801`7e3ead94 : 00000000`6d4e6f49 fffff801`7df6f06d ffffab8c`00000003 00000000`00000000 : nt!FsRtlQueryOpen+0xd1 ffffab8c`3d1de710 fffff801`7e1e62ba : fffff801`00000007 fffff801`7e1e5944 ffffab8c`3d1de950 00000000`00000000 : nt!IopQueryInformation+0x139ad4 ffffab8c`3d1de770 fffff801`7e1ecfcf : ffffc18d`dac1e8f0 ffffc18d`dac1e844 ffffc18d`f17f3010 00000000`00000000 : nt!IopParseDevice+0x8ea ffffab8c`3d1de8e0 fffff801`7e1eb431 : ffffc18d`f17f3000 ffffab8c`3d1deb28 ffffc18d`00000240 ffffc18d`cfcfe640 : nt!ObpLookupObjectName+0x78f ffffab8c`3d1deaa0 fffff801`7e457ec3 : 00000000`00000001 00000000`00000000 ffffab8c`3d1df090 ffffab8c`3d1deef8 : nt!ObOpenObjectByNameEx+0x201 ffffab8c`3d1debe0 fffff801`81d18063 : ffffab8c`3d1df000 ffffc18d`ed6ad9f0 ffffc18d`dd104a30 fffff801`81d076fb : nt!IoQueryInformationByName+0x263 ffffab8c`3d1dee90 fffff801`7af85c99 : ffffab8c`3d1df088 00000000`00000000 ffffab8c`3d1df088 fffff801`7dc6b455 : FLTMGR!FltQueryInformationByName+0x153 ffffab8c`3d1def40 fffff801`7af77924 : ffffab8c`3d1df088 00000000`00000000 00000000`00000000 00000000`00000000 : cldflt!FltQueryInformationByNameCallout+0x49 ffffab8c`3d1def90 fffff801`7afcf77d : 00000000`00000000 ffffab8c`3d1e0000 ffffab8c`3d1d9000 ffffc18d`ed6ad9f0 : cldflt!HsmExpandKernelStackAndCallout+0x44 ffffab8c`3d1defd0 fffff801`7afd0019 : ffffffff`0000ffff ffffc18d`f6666b38 ffffc18d`ed6adc80 ffffab8c`3d1df219 : cldflt!HsmiFltPreECPCREATE+0x34d ffffab8c`3d1df140 fffff801`81d04a5d : ffffc18d`f66669b0 00000000`00000000 00000000`00000000 00000000`00000000 : cldflt!HsmFltPreCREATE+0x9 ffffab8c`3d1df170 fffff801`81d045a0 : ffffab8c`3d1df2f0 ffffab8c`3d1df300 00000000`00000000 00000000`00000000 : FLTMGR!FltpPerformPreCallbacks+0x2fd ffffab8c`3d1df280 fffff801`81d3cd13 : fffff801`81d29060 00000000`00000090 00000000`00000000 00000000`000003a4 : FLTMGR!FltpPassThroughInternal+0x90 ffffab8c`3d1df2b0 fffff801`7dc31f39 : 00000000`00000000 fffff801`7e1e5905 00000000`00000000 00000000`00000000 : FLTMGR!FltpCreate+0x2f3 ffffab8c`3d1df360 fffff801`7dc30fe4 : 00000000`00000003 00000000`00000000 00000000`00000000 fffff801`7dc317a3 : nt!IofCallDriver+0x59 ffffab8c`3d1df3a0 fffff801`7e1e5ffb : ffffab8c`3d1df660 fffff801`7e1e5905 ffffab8c`3d1df5d0 ffffc18d`f63584e0 : nt!IoCallDriverWithTracing+0x34 ffffab8c`3d1df3f0 fffff801`7e1ecfcf : ffffc18d`dac1e8f0 ffffc18d`dac1e805 ffffc18d`f3973260 00000000`00000001 : nt!IopParseDevice+0x62b ffffab8c`3d1df560 fffff801`7e1eb431 : ffffc18d`f3973200 ffffab8c`3d1df7a8 00000000`00000040 ffffc18d`cfcfe640 : nt!ObpLookupObjectName+0x78f ffffab8c`3d1df720 fffff801`7e230300 : 00000000`00000001 000000e7`aa3dd4c8 00000000`00000001 00000000`00000000 : nt!ObOpenObjectByNameEx+0x201 ffffab8c`3d1df860 fffff801`7e22fac9 : 000000e7`aa3dd470 00000004`c0100080 000000e7`aa3dd4c8 000000e7`aa3dd488 : nt!IopCreateFile+0x820 ffffab8c`3d1df900 fffff801`7ddd2b15 : 00000000`00000000 00000000`00000000 00000000`00000000 000000e7`aa3dcb98 : nt!NtCreateFile+0x79 ffffab8c`3d1df990 00007ffe`7733cb64 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : nt!KiSystemServiceCopyEnd+0x25 000000e7`aa3dd3f8 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : 0x00007ffe`7733cb64 FOLLOWUP_IP: cldflt!HsmiFltPostECPCREATE+1da fffff801`7afcfbfe f60201 test byte ptr [rdx],1 SYMBOL_STACK_INDEX: 0 SYMBOL_NAME: cldflt!HsmiFltPostECPCREATE+1da FOLLOWUP_NAME: MachineOwner MODULE_NAME: cldflt IMAGE_NAME: cldflt.sys DEBUG_FLR_IMAGE_TIMESTAMP: 0 STACK_COMMAND: .cxr 0xffffab8c3d1dd800 ; kb BUCKET_ID_FUNC_OFFSET: 1da FAILURE_BUCKET_ID: 0x3B_cldflt!HsmiFltPostECPCREATE BUCKET_ID: 0x3B_cldflt!HsmiFltPostECPCREATE ANALYSIS_SOURCE: KM FAILURE_ID_HASH_STRING: km:0x3b_cldflt!hsmifltpostecpcreate FAILURE_ID_HASH: {ff49d151-c405-fda0-3953-8b48357a52e0} Followup: MachineOwner --------- 2: kd> lmvm cldflt start end module name fffff801`7af70000 fffff801`7afe7000 cldflt (pdb symbols) C:\ProgramData\dbg\sym\cldflt.pdb\0698036E8827B2FF6ECB6676372B81FC1\cldflt.pdb Loaded symbol image file: cldflt.sys Image path: \SystemRoot\system32\drivers\cldflt.sys Image name: cldflt.sys Timestamp: ***** Invalid (B7D0F1F2) CheckSum: 00079A82 ImageSize: 00077000 Translations: 0000.04b0 0000.04e4 0409.04b0 0409.04e4
Monday, October 14, 2019 7:05 AM -
User-460007017 posted
Hi fw-flw,
cldflt.sys belong to cloud file mini filter driver. Since the error message is thrown from native code. Could you fix this issue by rolling back these updates? I believe this issue is a compatibility issue.
I'm afraid you could only accept rolling back update as a workaround.
If you need to fix this, You may need to support a ticket to Microsoft since PG need to collect business impact.
If the reply is helpful, it is appreciated if you could mark the reply as answer.
Best Regards,
Jokies Ding
- Marked as answer by Anonymous Tuesday, September 28, 2021 12:00 AM
Monday, October 14, 2019 9:08 AM -
User515338469 posted
Could you please give me a direct link to a website where I can submit a support ticket to Microsoft? I am finding it difficult to get past all the bullshit (such as knowledge bases, chatbots, etc.) Microsoft set up to keep people from submitting support tickets. This is obviously something that needs to be looked at by a human with some expertise and access to Microsoft developers.
Otherwise, thanks for the help so far. As a workaround, we are now doing what we should have been doing for a while, which is replacing our home-brew IIS management application with appcmd.
Monday, October 14, 2019 9:33 AM -
User-460007017 posted
Hi fw-flw,
If you need to contact Professional Microsoft support engineer, you could create a business request ticket from here:
https://support.microsoft.com/en-us/supportforbusiness/productselection
Best Regards,
Jokies Ding
Tuesday, October 15, 2019 1:42 AM -
User515338469 posted
Thank you for your help.
I have marked your reply as answer. We will get in touch with Microsoft support.
Tuesday, October 15, 2019 7:27 AM