none
OutofMemory Exception problem. RRS feed

  • Question

  • I am trying to write a wcf service that receives large file uploads I am using mtom and streaming to create the service upload works fine the file  uploads correctly to the service and I can output this to the file system. The problem arises when I process the file so it conforms to a third party IT  system which needs the file inserted into the database after it has been base64 encoded this increases the size of the uploaded file massively and causes  an outofmemory exception. I have increased the size of all settings for the service to maximum including MaxBufferSize, MaxBufferPoolSize

     

      <system.web>

        <compilation debug="true" />

        <httpRuntime executionTimeout="43210000" maxRequestLength="2097151" />  

      </system.web>

      <!-- When deploying the service library project, the content of the config file must be added to the host's

      app.config file. System.Configuration does not support config files for libraries. -->

      <system.serviceModel>

        <bindings>

          <basicHttpBinding>

            <binding name="FileTransferServicesBinding"

           transferMode="Streamed"

           messageEncoding="Mtom"

     maxBufferSize="2147438647" maxBufferPoolSize="9223372036854775807"

              maxReceivedMessageSize="9223372036854775807">

              <readerQuotas maxDepth="2147438647" maxStringContentLength="2147438647"

                maxArrayLength="2147438647" maxBytesPerRead="2147438647" maxNameTableCharCount="2147438647" />

              <security mode="None" />

            </binding>

          </basicHttpBinding>

        </bindings>

     

    And I still get OutofMemory Exception error I have also increased the memorylimit in processmodel section of the machine.config to 100%.

     

    I realise having these settings at max is not recommended and when I trace the real cause of the memory exception I will bring these values down to more  sensible levels.

     

    Please can anyone suggest what might be causing this memory exception or some steps I could try to trace the problem.

     

    Size of the file I am testing against is 184 MB (193,022,464 bytes)

     

    Code in service

     

     

                    // Now read s into a byte buffer.

                    byte[] myData = new byte[request.Length];

                    int numBytesToRead = (int)request.Length;

                    int numBytesRead = 0;

                    while (numBytesToRead > 0)

                    {

                        // Read may return anything from 0 to numBytesToRead.

                        int n = request.FileByteStream.Read(myData, numBytesRead, numBytesToRead);

                        // The end of the file is reached.

                        if (n == 0)

                            break;

                        numBytesRead += n;

                        numBytesToRead -= n;

                    }

                    request.FileByteStream.Close();

     

                    string myFile64String = Convert.ToBase64String(myData, 0, myData.Length);

     

                    myData = null;

     

                    byte[] myFile64StringByte = StrToByteArray(myFile64String);

     

                    string SystemGuid = ConfigurationManager.AppSettings["Guid"];

                    string ContentsEndString = "";

                    int myFileLength = myFile64StringByte.Length;

                    byte[] DataToInsertByte = new byte[myFileLength + 78]; // Make an array the size of the file contents + 76 for the GUID prefix and + 2  for the empty characters at end of file

                    byte[] SystemGuidByte = StrToByteArray(SystemGuid);

                    byte[] ContentsEndStringByte = StrToByteArray(ContentsEndString);

    Thursday, June 4, 2009 8:30 AM

Answers

  • Hi,

    Could you tell me the total memory size of the server hosting WCF? When this exception is thrown could you check the memory used by the process hosting WCF? How many memories are occupied by this process? What's the memory usage at that moment? What's the stack trace when this exception is thrown?

    You can check your code to see if there's any code may cause memory leak. (Try to comment all the code in operation contract to see whether it's WCF's issue or caused by your code)

    To troubleshoot this issue please refer to:

    http://msdn.microsoft.com/en-us/library/ms954591.aspx
    http://www.eggheadcafe.com/articles/20060114.asp
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.
    Monday, June 8, 2009 10:11 AM
    Moderator