none
Bug in NTFS Telemtry code causing a BSOD

    Question

  • Hello,

    We have ran into an issue where a system BSOD and the bug seems to be a bug in NTFS I/O telemetry code.

    In our filter driver we issue an IRP read via FltReadFile with the flags:
    FLTFL_IO_OPERATION_SYNCHRONOUS_PAGING | FLTFL_IO_OPERATION_PAGING

    We do this in certain cases in our PreClose callback, it seems like there is a code in NTFS I/O telemetry which
    attempts to do ObReferenceObject on the FILE_OBJECT even though it's not allowed in this specific flow since
    the IO Manager already took the FILE_OBJECT ReferenceCount to 0.

    As a workaround for now we will be changing the Thread I/O priority to Low prior to issuing the paging I/O reads
    during PreClose, and then revert it back to it's original I/O priority (this only happens if the original Thread I/O priority
    is higher than Low)

    Here is a sample call stack:
    nt!KeBugCheckEx
    nt!ObfReferenceObject+0x1306c3
    Ntfs!NtfsTelemetryPostFileObjectInfo+0xd5
    Ntfs!NtfsTelemetryCollectReadWritePerfData+0x245
    Ntfs!NtfsExtendedCompleteRequestInternal+0x1e2
    Ntfs!NtfsCommonRead+0x21a1
    Ntfs!NtfsFsdRead+0x1d8
    nt!IofCallDriver+0x59
    FLTMGR!FltpLegacyProcessingAfterPreCallbacksCompleted+0x157
    FLTMGR!FltPerformSynchronousIo+0x2c9
    FLTMGR!FltReadFileEx+0x189
    FLTMGR!FltReadFile+0x51
    myfltdrv!XXX1+0x215
    myfltdrv!XXX2+0x2df
    myfltdrv!XXX3+0x8e
    myfltdrv!FsPreClose+0xe2
    FLTMGR!FltpPerformPreCallbacks+0x2dc
    FLTMGR!FltpPassThroughInternal+0x8c
    FLTMGR!FltpPassThrough+0x144
    FLTMGR!FltpDispatch+0x9e
    nt!IofCallDriver+0x59
    nt!IopDeleteFile+0x124
    nt!ObpRemoveObjectRoutine+0x80
    nt!ObfDereferenceObject+0xa1
    nt!MiSegmentDelete+0x192
    nt!MiProcessDereferenceList+0xa3
    nt!MiDereferenceSegmentThread+0x125
    nt!PspSystemThreadStartup+0x47
    nt!KiStartSystemThread+0x16

    There is a similar issue with fileinfo.sys
    nt!KeBugCheckEx
    nt!ObfReferenceObject+0x8c90f
    fileinfo!FIStreamQueueQuery+0x9b
    fileinfo!FIPreReadWriteCallback+0x148
    FLTMGR!FltpPerformPreCallbacks+0x2ea
    FLTMGR!FltpPassThroughInternal+0x88
    FLTMGR!FltPerformSynchronousIo+0x2f4
    FLTMGR!FltReadFileEx+0x455
    FLTMGR!FltReadFile+0x51
    myfltdrv!XXX1+0x226
    myfltdrv!XXX2+0x2c0
    myfltdrv!XXX3+0x4f
    myfltdrv!FsPreClose+0x88
    FLTMGR!FltpPerformPreCallbacks+0x2ea
    FLTMGR!FltpPassThroughInternal+0x88
    FLTMGR!FltpPassThrough+0x1a6
    FLTMGR!FltpDispatch+0x9e
    nt!IopDeleteFile+0x12d
    nt!ObpRemoveObjectRoutine+0x78
    nt!ObfDereferenceObjectWithTag+0xc6
    nt!ObCloseHandleTableEntry+0x28b
    nt!NtClose+0xcb
    nt!KiSystemServiceCopyEnd+0x13 (TrapFrame @ ffff9600`9486fb00)
    ntdll!NtClose+0x14
    KERNELBASE!CloseHandle+0x62
    mssph!CFileHandler::CloseAccessor+0xd2
    SearchProtocolHost!UrlAccessorPointer::~UrlAccessorPointer+0x3d
    SearchProtocolHost!CFilterThread::Thread+0x23f8
    KERNEL32!BaseThreadInitThunk+0x14
    ntdll!RtlUserThreadStart+0x21

    Now according to the documentation it is allowed to send paging I/O IRP during PreClose as far as we saw.

    In order to replicate the issue with FileInfo you will need one thread sending the paging I/O reads during PreClose on
    let's say FILE_OBJECT_A.
    While another thread working with a different FILE_OBJECT_B (notice FileInfo manages everything in a StreamContext,
    so basically FILE_OBJECT_A and FILE_OBJECT_B point to the same StreamContext) and this second thread is going
    to issue a rename via SetFileInformation.

    Basically it's just a race in which fileinfo!FIPreReadWriteCallback will attempt to acquire the new name before the second
    thread because of the way the information is managed in the StreamContext...

    Thanks,
    -- Jon.
    Monday, February 11, 2019 7:21 AM

All replies

  • Which release of windows 10? Your best bet is to open a support incident

    d -- This posting is provided "AS IS" with no warranties, and confers no rights.

    Monday, February 11, 2019 5:22 PM
  • Hi Doron,

    Thank you for the reply, this happened on the latest Windows 10 RS5, I did try to open a support case but i got a reply to post the issue here in  Windows Hardware WDK and Driver Development

    Tuesday, February 12, 2019 5:54 AM