locked
FileSystemWatcher:Changed behavior from 2003 R2 to 2008 R2 (clients also) RRS feed

  • Question

  • Hi All !

    I want to find out, if the "new behavior" of the FileSystemWatcher [FSW] is just planned, a bug in Net or Windows Server 2008 R2/W7.

    I have a long time running app, which monitors logfiles. There were no code changes, when I moved it to Server 2008 R2, but the behavior differs significantly.

    Some apps are using this behavior to write logfiles:

    Open/append the file, keep the file open, log from time to time.

    In Server 2003 R2 [Frameworks 2/4] each logentry into the mentioned logfiles causes the FSW to raise a "changed" event.

    This is not the case for Server 2008 R2/w7.

    With this behvior, I can put the app to the garbage. BTW, I have verified, that this behavior is on all Server 2008 R2, not a single one. The app was in use for years now and no code changes were applied.

    This looks like a bug, ether in the Framework, but more likely in Server 2008 R2 itself. Nothing is written about a changed behavior of the FSW.

    The only options I currently see, is, to downgrade the server to Server 2003 R2 or migrate the logfiles (and/or the server) to a Linux store. In both cases, everything works like expected.

    Some hints or tips and probably a link the a statement, which states

    changed behavior are really very welcome.

    Thanks so far and

    best regards,

    ++mabra

    Saturday, January 19, 2013 4:00 PM

All replies

  • Hi Mabra,

    Welcome to the MSDN Forum.

    Please report this issue on Connect site:http://connect.microsoft.com/VisualStudio/  

    When you finished, post the link here, please.

    Thank you.

    Best regards,


    Mike Feng
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Monday, January 21, 2013 9:33 AM
  • "This looks like a bug, ether in the Framework, but more likely in Server 2008 R2 itself. Nothing is written about a changed behavior of the FSW."

    And nothing is written about a change even being raised every time a log entry is written to the file. It's not a bug in either the framework or Windows because neither of them ever documented precisely when a change event is raised.

    Monday, January 21, 2013 9:49 AM
  • Hi !

    Ok, thanks;I'll go that route.

    br++mabra

    Monday, January 21, 2013 2:30 PM
  • Hi !

    Ok, the statements about the trackinga are really a litte vague ... But why would an EventFilter for "Size" exists, if it should not fire .... ?  ;-)

    And: Do you have regarded the fact, that this worked for years for me ?? Not so on Server 2008 R2. Again: There were no code changes applied ....

    BTW, on my box - probably from the Unix-Tools - is a program, "tail.exe". Astoundingly this works, if the logging program appends a line .... Linux to the rescue ???

    br++mabra




    • Edited by mabra99 Monday, January 21, 2013 2:36 PM
    Monday, January 21, 2013 2:34 PM
  • " But why would an EventFilter for "Size" exists, if it should not fire .... ?  ;-)"

    It will fire when the file being written to is closed. It may fire earlier than that but that depends on how OS feels like, generating a lot of file changed events is not exactly a recipe for performance.

    "Do you have regarded the fact, that this worked for years for me ?? Not so on Server 2008 R2. Again: There were no code changes applied ...."

    But the OS changed. Why would there be a new OS version if nothing ever changes?

    "BTW, on my box - probably from the Unix-Tools - is a program, "tail.exe". Astoundingly this works, if the logging program appends a line .... Linux to the rescue ???"

    tail may very well use polling to detect file changes.

    Monday, January 21, 2013 3:16 PM
  • Hi !

    I agree, that this happens on another OS. But nothing documents a changed

    behavior for the FSW. I really would like to prevent me to poll!

    br++mabra

    Monday, January 21, 2013 4:26 PM
  • As already mentioned, there may be changes in the file system that causes the FSW to behave differently.

    What NotifyFilter are you using?

    Monday, January 21, 2013 9:33 PM
  • Hi !

    May be, but this disables my app completely and I expect, that this behavioral changes should be documented somewhere ...

    I filed a bug at msconnect ...

    My filter:

    LastWrite | LastAccess | Size

    So, not only size ...

    Works like a charm on 2003/XP.

    br,

    ++mabra

    Friday, January 25, 2013 12:29 AM
  • Hi Mabra,

    Would you like to post the Connect link here?

    Thank you.

    Best regards,


    Mike Feng
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Friday, January 25, 2013 7:03 AM
  • Hi !

    Ok, this is it:

    ms-connect-question

    br,

    ++mabra

    • Marked as answer by Mike Feng Monday, January 28, 2013 4:48 AM
    • Unmarked as answer by mabra99 Thursday, February 21, 2013 3:40 PM
    Friday, January 25, 2013 8:06 AM
  • Hi All !

    Who is interested, I - indirectly - got a confirmation, that this is a bug, but they have other priorities ....

    and it was set to "Won't Fix" today.

    :-)

    I am completely disappointed now.

    Best regards,

    ++mabra

    Thursday, February 21, 2013 3:43 PM
  • This is with .NET 2.0? Have you tried with 4.0?


    EDIT: It seem you already did from the original post...
    Thursday, March 7, 2013 9:32 PM
  • Oh, come on, this doesn't look like a bug, more like a change in OS file caching.

    Documentation clearly says that "any file-size (and last-write-time as well) change in the watched directory or subtree causes a change notification wait operation to return. The operating system detects a change in file size only when the file is written to the disk. For operating systems that use extensive caching, detection occurs only when the cache is sufficiently flushed."

    So most likely Win 2008 R2, unlike 2003 R2, holds more data in cache and flushes it rarely.

    Saturday, March 9, 2013 3:59 PM
  • Hi All !

    Explicit no: It happens with Net 2 AND Net 4 [I wrote this]. I can see,

    that the caching for the OSses may have been changed, but the [logical]contract of the Net API should be consistent and independent from a concrete OS version.

    Otherwise the changed behavior must be documented and it is not. The statement that "they have other oriorities to fix it in the next relases" shows, that it is seen as a bug.

    So, I'll look how to survife until Net 6.

    Best regards,

    ++mabra



    • Edited by mabra99 Saturday, March 9, 2013 4:51 PM
    Saturday, March 9, 2013 4:50 PM
  • This has nothing to do with .NET, so I doubt .NET 6 would save you. FSW solely relies on FindFirst(Next)ChangeNotification under the hood, and that is system API. There's no other documented way to monitor file changes.

    I've just remembered that the other significant change since Vista/2008 is that LastAccessDate updating is disabled on NTFS volumes by default. You may try to enable it back and see if that helps somehow.

    Saturday, March 9, 2013 5:09 PM
  • Hi !

    Sorry, I disagree. Net is the platform service.

    If there is a decrepancy between Win32 API and the Net API, it must be documented. The API, each, is an abstraction to use OS features. So if Net gives you an API description, it must be fullfilled. It is not. Otherwise, reading API specs and programming against it would become total arbitrarily.

    BTW, I tried to enable the LastAccessUpdate, but it does'nt change the behavior.

    br,

    ++mabra



    • Edited by mabra99 Saturday, March 9, 2013 5:20 PM
    Saturday, March 9, 2013 5:19 PM
  • I've made a small research, and it really appears a heavy caching side effect related neither to FileSystemWatcher class nor to .NET framework.

    Writing to file in Win7/2008 doesn't commit to disk at once, even if you flush data (calling either fflush() in C or FileStream.Flush() in .NET), so FSW doesn't know file has changed and is not triggering Change event.

    And this doesn't break a logical contract for FSW – it always raises an event when changes are written to disk. It might not be clearly stated in the .NET documentation, but the moment when file buffers are flushed to disk is actually the moment when its file system entry is updated (i.e. "size" property changes), so FSW behavior is correct here.

    To override caching issues, you may try to disable write cache on OS level (not worked for me though), or change how the programs are writing to file:

    1. If your app is .NET app, call FileStream.Flush(true) overload (available from .NET 4), or FlushFileBuffers() WinAPI with native file handle.

    2. If your app is C app, either open files with "c" flag (i.e. open("file.log", "wc")), or call FlushFileBuffers() WinAPI function after write, or link app with commode.obj.

    Also, opening an open file for read in another app (for example, refreshing directory listing in Explorer, or viewing your log in Notepad) causes flushing cached buffers and triggering events on FSW.

    Saturday, March 9, 2013 8:22 PM
  • Hi !

    Much thanks for your investigation.

    You might have been seen, that I uploaded a repro to MS-Connect [the app is a .net app].

    The VS team closed the issue, after they stated "no time/not on our prio list".

    I noticed and wrote already, that "flush" does'nt help.

    I know, that openening that file helps [FSW then fires the event].

    But this would lead to an app, which can completely be written without the FSW, because I have to poll each monitored file [by an open/close cycle].

    I am monitoring something between 15.000-25.000 files a day. Works fine in server 2003 and Linux, but not on server 2008.

    That the caching behavior might [I am sure] have changed does not help me.

    br,

    ++mabra


    • Edited by mabra99 Sunday, March 10, 2013 9:26 AM
    Sunday, March 10, 2013 9:26 AM