Paging, MemoryMappedFile and GC fun RRS feed

  • Question

  •  I have 16GB of data collecting in memory, in a 64-bit c# project using MemoryMappedFile.  The data arrives from a network connection on quite a quick transport (a good chunk of data arriving every 200ms), and I write it into a view of that memory mapped file.   The system has 32BG of ram, and my assemblies are all marked LARGEADDRESSAWARE, so everything is capable of keeping the 16GB of data in memory at all times.  (It's a 6 core Windows Server 2012R2 x64 server...)

    What I'm observing is that the GC runs eventually and it appears to trim the process's working set.  This causes a massive page-out of data to the disk.  This massive disk I/O in turn causes any writing to the memory backed by the file to stall, and I lose my 200ms timings, falling way behind quickly.  (In some cases, more than a couple of seconds, and the disks are pegged)

    I've observed it via PerfMon and VMMap - pages slowly build in my working set until the GC runs, and then the MemoryMappedFile's pages are mostly trimmed - some of them down to 1MB in size.  They then have to grow back up as new data arrives, and the process repeats again.

    I've tried setting Process.MaxWorkingSet and Process.MinWorkingSet and see minute differences, but it seems the same thing happens, just a but later.  (Both were set to the size of my memory mapped file, plus some overhead) I've also tried p/invoking SetWorkingetSizeEX with QUOTA_LIMITS_HARDWS_MIN_ENABLE.  Does anyone have any ideas how I can lock the pages from a C# MemoryMappedFile's view in memory OR prevent the GC from trimming the working set?  I'd like to have the full map in RAM and only page out changes to each dirty page (which should be acceptable in terms of timing/performance)

    Darin R.

    Thursday, February 19, 2015 5:15 PM


  • It sounds like you need some VirtualLock calls, not sure what objects you want to lock, but if you want to get your hands into CLR's memory management you need a CLR host (probably written in C++).

    Visual C++ MVP

    • Marked as answer by Fred Bao Monday, March 2, 2015 10:01 AM
    Thursday, February 19, 2015 8:26 PM