none
How can we reduce the consumption rate of the UWF overlay? RRS feed

  • Question

  • Our appliance running Embedded 8.1 Industry has consumed its entire 1024 GB Unified Write Filter overlay in less than 24 hours.  Where can we find a resource with Windows configuration guidelines for reducing this consumption rate?  The high rate that we are seeing makes it impossible to use UWF on our appliance, which must function for weeks at a time with no downtime or reboots.  UWF was one of the reasons that we chose Windows Embedded.
    Wednesday, October 21, 2015 8:58 PM

All replies

  • Use UWFMGR.exe and the OverlayDetail Option to get a list of items (files) that are in the overlay. Once you have this list then you can take corrective action to open holes or other.

    Sean Liming - Book Author: Starter Guide SIM (WEI), Pro Guide to WE8S & WES 7, Pro Guide to POS for .NET - www.annabooks.com / www.seanliming.com

    Wednesday, October 21, 2015 11:00 PM
    Moderator
  • Thank you for specifying that very useful command.

    By "open holes" do you mean creating file / folder exceptions to the write filter?  Once this is done, wouldn't the overlay still continue to fill up even with the exceptions due to the bypass module having to allocate traffic?  I have been told that this causes the overlay to fill based on the number of exclusions and transactions that must take place on the disk.
    Friday, October 23, 2015 6:21 PM
  • Yes I mean file and folder exclusions. Someone external asked me your question. I saw what was written in the MS notes, but the wording doesn't make sense. What ever you open up an exclusion, data should be written directly to the disk and not effect the overlay. The only issues is the size of the file. Very big files in the GB don't work well with FBWF.

    Sean Liming - Book Author: Starter Guide SIM (WEI), Pro Guide to WE8S & WES 7, Pro Guide to POS for .NET - www.annabooks.com / www.seanliming.com

    Sunday, October 25, 2015 11:23 PM
    Moderator
  • Wow; OK.  What you just said about writing to excluded files / folders not filling up the overlay makes more sense to me than the Microsoft answer that was forwarded to me.  Also, it will allow us to move forward and continue using the write filter with the right exclusions without worrying about the overlay filling up.
    Tuesday, October 27, 2015 5:33 PM
  • You still need to check to see what files that are not in your direct control that might fill up the overlay.

    Sean Liming - Book Author: Starter Guide SIM (WEI), Pro Guide to WE8S & WES 7, Pro Guide to POS for .NET - www.annabooks.com / www.seanliming.com

    Tuesday, October 27, 2015 5:36 PM
    Moderator
  • Have you made any progress on this? I just started working on making UWF actually behave (And play nice with Intel McAfee Application Control) so I'm interested.

    uwfmgr overlay get-files c:

    shows a remarkable number of <windowsystuff>::$DATA   <-- actual files being written I think.

    which is... a problem at 4K each. I'm up to about 70 MB after an hour of doing nothing...


    =^x^=


    • Edited by P 'Bunny' E Monday, November 9, 2015 9:34 AM gronk
    Monday, November 9, 2015 4:10 AM
  • I have the same problem, I have set up the limit for the RAM based overlay to 2048 MB.

    After few hours of doing nothing, I check the overlay status with: uwfmgr overlay get-consumption

    result: The overlay consumption is 1580 MB.  ?!

    Why?

    I have do nothing, and this is a standard wei 8.1 pro image installed, without any modifications.

    If nobody know where is the solution to this problem, I suppose I need to have 16GB or 32Gb RAM memory installed, if I want that my pc can run 12-15 hours a day, without warnings.

    Monday, November 30, 2015 3:36 PM
  • Run the UWFMGR /overlaydetail to get a list of what is being put into the overlay,

    Sean Liming - Book Author: Starter Guide SIM (WEI), Pro Guide to WE8S & WES 7, Pro Guide to POS for .NET - www.annabooks.com / www.seanliming.com

    Monday, November 30, 2015 4:56 PM
    Moderator
  • Some possible candidates:

    • Pagefile (just to mentoin, I hardly believe you know that)
    • System Restore
    • Prefetching
    • Indexing
    • Temporary (Internet files)

    Tuesday, December 1, 2015 7:38 AM
  • My money says it is his application. I look forward to seeing the result.

    I disable the pagefile on my WES8 boxes since I have 8Gb of ram and coming anywhere near the endstops is a failure situation.


    =^x^=

    Wednesday, December 2, 2015 3:33 AM
  • Is it better to enable or disable Pagefile?

    Here is the pictures what I have when I check the overlay consumption in few hours doing nothing:

    https://www.dropbox.com/s/ru3dw3pv19yubf1/Photo%2002.12.15%2016%2036%2042.jpg?dl=0

    And here is 3 pictures from: uwfmgr overlay get-consumption c:  (uwfmgr /overlaydetail is not working)

    1)

    https://www.dropbox.com/s/44i7oq2nt3m42lq/Photo%2002.12.15%2016%2058%2009.jpg?dl=0

    2)

    https://www.dropbox.com/s/ggdca3ij4piqbci/Photo%2002.12.15%2016%2040%2050.jpg?dl=0

    3)

    https://www.dropbox.com/s/vn5eqbyrb288as4/Photo%2002.12.15%2016%2040%2040.jpg?dl=0

    Thursday, December 3, 2015 9:33 AM
  • It depends on your memory. I have it disabled because I have 8Gb and my overlay is set for 1Gb warning at 940Mb or so. If it is on the same drive you are protecting, you will need to exclude stuff.

    Unfortunately, your screen shots don't show where all the space is going.

    Can you try the following: (Obviously change d:\temp\ for your USB memory whatever.

    uwfmgr get-config >d:\temp\out.txt
    uwfmgr overlay get-consumption >>d:\temp\out.txt
    uwfmgr overlay get-files c: >>d:\temp\out.txt

    ------------------

    For what it is worth, my new STB has been running for 20 hours and is up to 343mb (rather more than I expected) get-files tells me that most of that is stupid .NET 4 garbage like:

    Unified Write Filter Configuration Utility version 6.3.9600
    Copyright (C) Microsoft Corporation. All rights reserved.
    
    The overlay consumption is 343 MB.
    Unified Write Filter Configuration Utility version 6.3.9600
    Copyright (C) Microsoft Corporation. All rights reserved.
    
    Searching overlay files of c:, please wait...
               Bytes File
              430080 \$BitMap::$DATA
                8192 \$Boot::$DATA
               16384 \$Extend\$ObjId:$O:$INDEX_ALLOCATION
                4096 \$Extend\$RmMetadata\$Txf:$I30:$INDEX_ALLOCATION
                4096 \$Extend\$RmMetadata\$TxfLog:$I30:$INDEX_ALLOCATION
               65536 \$Extend\$RmMetadata\$TxfLog\$TxfLog.blf::$DATA
              245760 \$Extend\$RmMetadata\$TxfLog\$TxfLogContainer00000000000000000003::$DATA
                8192 \$Extend\$RmMetadata\$Txf\000000000000474F::$DATA
                8192 \$Extend\$RmMetadata\$Txf\0000000000004750::$DATA
             1572864 \$Extend\$UsnJrnl:$J:$DATA
            35127296 \$LogFile::$DATA
               16384 \$Mft::$BITMAP
             4194304 \$Mft::$DATA
                4096 \$MftMirr::$DATA
               20480 \$Secure:$SDH:$INDEX_ALLOCATION
               36864 \$Secure:$SDS:$DATA
    
    
    
                4096 \Windows\assembly\NativeImages_v4.0.30319_64\System.Manaa57fc8cc#\05c9dba17e25db901ba22a7094769b2b:$I30:$INDEX_ALLOCATION
                4096 \Windows\assembly\NativeImages_v4.0.30319_64\System.Manaa57fc8cc#\05c9dba17e25db901ba22a7094769b2b\System.Management.Automation.ni.dll.aux::$DATA
            30916608 \Windows\assembly\NativeImages_v4.0.30319_64\System.Manaa57fc8cc#\05c9dba17e25db901ba22a7094769b2b\System.Management.Automation.ni.dll::$DATA
                4096 \Windows\assembly\NativeImages_v4.0.30319_64\System.Management\6a52de0c25736e83ea8c4b85f6a38fca\System.Management.ni.dll.aux::$DATA
                4096 \Windows\assembly\NativeImages_v4.0.30319_64\System:$I30:$INDEX_ALLOCATION
    
    
                4096 \Windows\assembly\NativeImages_v4.0.30319_64\System.Manaa57fc8cc#\05c9dba17e25db901ba22a7094769b2b:$I30:$INDEX_ALLOCATION
                4096 \Windows\assembly\NativeImages_v4.0.30319_64\System.Manaa57fc8cc#\05c9dba17e25db901ba22a7094769b2b\System.Management.Automation.ni.dll.aux::$DATA
            30916608 \Windows\assembly\NativeImages_v4.0.30319_64\System.Manaa57fc8cc#\05c9dba17e25db901ba22a7094769b2b\System.Management.Automation.ni.dll::$DATA
                4096 \Windows\assembly\NativeImages_v4.0.30319_64\System.Management\6a52de0c25736e83ea8c4b85f6a38fca\System.Management.ni.dll.aux::$DATA
                4096 \Windows\assembly\NativeImages_v4.0.30319_64\System:$I30:$INDEX_ALLOCATION
    


    =^x^=

    Friday, December 4, 2015 1:08 AM
  • I also have 8GB RAM, and pagefile is enabled.

    RAM overlay is set to 2048 MB.

    Here is the get-config, get-consumption, get-files :

    https://www.dropbox.com/sh/izkpnchzfp6bsqo/AAAl9FTMjBO-QYHGcBYKMnQPa?dl=0

    How can I reduce all this stuff?

    Friday, December 4, 2015 9:27 AM
  • Excellent!

    You have no exceptions for windows, which means you're going to be in trouble as windows does all sorts of pointless stuff and you're caching it in ram.

    I can see you have the same thing as me, .NET pointless garbage (I haven't had time to look at this yet.)

    You have about 4* as many system files being touched as my image does in 20 hours of uptime, which suggests to me that you have unwanted services running.

    I did a quick side by side compare.

    you're writing to C drive a lot more than I am. metadata (first 20 or so lines) adds up to 140MB (vs 36)

    Program files, about 5* as much activity.

    Program data, you need to exclude windows defender or kill it.

    Users, store should be disabled for embedded stuff. It's in the guidelines for WES8.

    You aren't running windows update or something are you? Because your WINSXS is 15* busier than mine.

    I think that's the important hint. Time to dig up the manual because that's all I did.

    I make simple scripts to disable tasks and services, cleanup everything after running windows update (against my own WSUS server) and things.


    =^x^=

    Friday, December 4, 2015 10:20 AM