none
StaticFileHandler and OutOfMemoryException when used with a custom VirtualPathProvider RRS feed

  • Question

  • I found that the use of the StaticFileHandler together with a custom VirtualPathProvider will cause an OutOfMemoryException when the file to be provided is too big, because it does not use the TrasmitFile API but allocates the entire buffer in memory.

    I found this article that says that an hotfix is available, but says to wait for a new ServicePack but I currently have the SP2:

    http://support.microsoft.com/kb/947461/en-us

    Does the SP2 contain this hotfix?

    There is an alternative to StaticFileHandler that avod the issue?


    chrjs

    Friday, May 24, 2013 1:12 PM

Answers

  • Hi Michael,

    thanks for the feedback.

    I can't figure out the reason (except considering that a bug) to allocate the buffer when the VPP is providing a generic stream: reading chunks from the stream and write them to the _httpWriter should solve the memory issue ad seems to me the correct implementation.

    I my case, the stream is a FileStream, but does not matter.

    As I understand, I must implement a custom Handler for that VPP to solve the memory issue.
    I hope anyway to see this changed in next versions.

    Christian.

    chrjs

    Monday, May 27, 2013 2:19 PM

All replies

  • .NET v2 is an old framework.  I'd recommend that you try it using the latest version of v3.5 and if it still occurs then you'll have to use an alternative.  The KB wasn't updated to include v3.x so I assume it is fixed.

    Michael Taylor
    http://msmvps.com/blogs/p3net

    Friday, May 24, 2013 2:01 PM
    Moderator
  • we are using .NET 3.5. I wrote 2.0 because the System.Web is still labeled 2.0.

    Opening the System.Web assembly with reflector seems that the issue is still there, because the flow goes to call response.WriteVirtualFile() that is defined as below and allocates a byte[] of the size of the VirtualFile:

    internal void WriteVirtualFile(VirtualFile vf)
    {
        using (Stream stream = vf.Open())
        {
            if (this.UsingHttpWriter)
            {
                long length = stream.Length;
                if (length > 0L)
                {
                    byte[] buffer = new byte[(int) length];
                    int count = stream.Read(buffer, 0, (int) length);
                    this._httpWriter.WriteBytes(buffer, 0, count);
                }
            }
            else
            {
                this.WriteStreamAsText(stream, 0L, -1L);
            }
        }
    }
    
     
    
     

    Here the call stack of the exception:

    System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
     at System.Web.HttpResponse.WriteVirtualFile(VirtualFile vf)
     at System.Web.StaticFileHandler.ProcessRequestForNonMapPathBasedVirtualFile(HttpRequest request, HttpResponse response)
     at System.Web.StaticFileHandler.ProcessRequestInternal(HttpContext context)
     at System.Web.StaticFileHandler.ProcessRequest(HttpContext context)
     at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
     at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) 


    chrjs


    • Edited by chrjs Friday, May 24, 2013 2:54 PM
    Friday, May 24, 2013 2:51 PM
  • I don't think the hotfix you are referring to solves the issue you're mentioning.  It mentions that you get the exception even though there is memory available.  The current version is still going to allocate an array so if you have a really large file then you're still likely to get an OOM exception because the memory has to be contiguous.  That is the way the implementation works. 

    I think you're going to have to go with an alternative implementation that doesn't read the file into memory as the current (v4.x) implementation will store the file in memory temporarily as it streams it out.  If you think about how virtual providers work this actually makes sense.  TransmitFile requires a physical file to exist.  A VPP doesn't necessarily store the file physically at all so TransmitFile cannot be used without first serializing the file to disk.  This can be expensive and time consuming so the implementation doesn't bother.  The alternative approach (which might be better) would be to stream the file across the wire from the VPP to the response rather than storing it in a temporary array first but the existing implementation doesn't do that. 

    Michael Taylor
    http://msmvps.com/blogs/p3net

    Friday, May 24, 2013 3:18 PM
    Moderator
  • Hi Michael,

    thanks for the feedback.

    I can't figure out the reason (except considering that a bug) to allocate the buffer when the VPP is providing a generic stream: reading chunks from the stream and write them to the _httpWriter should solve the memory issue ad seems to me the correct implementation.

    I my case, the stream is a FileStream, but does not matter.

    As I understand, I must implement a custom Handler for that VPP to solve the memory issue.
    I hope anyway to see this changed in next versions.

    Christian.

    chrjs

    Monday, May 27, 2013 2:19 PM