none
Out of Memory error. Need help and ideas RRS feed

  • Question

  • Hi all,

    I have a Background Worker routine that reads a blob data type from an MS Access database and saves it as a file. The 'DoWork' routine reads all the records and processes them in an OLEDbDataReader 'while(Read)' loop, calling the 'progresschanged' method for each row. This has worked every time except once. In the 'progresschanged' method of the background worker I stuff the bytes into a memory stream, create an image from the memory stream show a preview and create a file. Could you look at the progresschanged code (Where the error was thrown) and suggest maybe a better way of doing this, and why? Or, how to keep up with how many files the OS is processing and call a sleep every so often? Thanks, here is the code.

                try
                {
                    if (e.UserState != null)
                    {
                        exportedImage img = (exportedImage)e.UserState;
                        if (img.RecordImage != null)
                        {
                            string uniqueTag = "";
                            lblImageName.Text = img.Name;
                            // Put the binary array data from the Blob field into a memory stream
                            using (MemoryStream ms = new MemoryStream(img.RecordImage))
                            {
                                // Move the memory stream into an image object for previewing
                                System.Drawing.Image imgpreview = System.Drawing.Image.FromStream(ms);
                                pictureBox1.Image = imgpreview;
                                // Update GUI
                                lblImageName.Refresh(); pictureBox1.Refresh();
                                // Save the image to a bmp file
                                uniqueTag = "_" + DateTime.Now.ToUniversalTime().Ticks.ToString();
                                imgpreview.Save(Path.Combine(m_ExportPath, img.Name + uniqueTag + ".bmp"));
                            }
                        }
                    }
                }
                catch(Exception ex)
                {
                    //Notify and Log
                    string msg = "An error occured while processing the image file. " +
                        "Please check the following message: " + Environment.NewLine + ex.Message.ToString();
                    MessageBox.Show(msg);
                    LogMessage("backgroundWorker1_ProgressChanged()", ex.Message.ToString());
                }

    Wednesday, September 27, 2017 8:13 PM

Answers

  • Hello mgppgm,

    >>Could you look at the progresschanged code (Where the error was thrown) and suggest maybe a better way of doing this, and why

    For your  code, the outofmemory exception occurs when create a lot of reference object result in memory leaking .

    using (MemoryStream ms = new MemoryStream(img.RecordImage))

    The following description from MSDN.

    when you use an IDisposable object, you should declare and instantiate it in a using statement. The using statement calls the Dispose method on the object in the correct way, and (when you use it as shown earlier) it also causes the object itself to go out of scope as soon as Dispose is called.

    In fact, the using statement are not much more than syntactic sugar for try-finally blocks.I have tested your code and the using statement seems that no object is released. I have optimized your code base on your description . the following code is my testing result.

    MemoryStream memoryStream;   
            private void backgroundWorker1_ProgressChanged(object sender, ProgressChangedEventArgs e)
            {
                string path = @"C:\Users\v-feih\Pictures\Saved Pictures\1.png";
                Byte[] s = GetPictureData(path);
                try
                {
                     memoryStream= new MemoryStream(s);
    
                    System.Drawing.Image imgpreview = System.Drawing.Image.FromStream(memoryStream);
                 
                    pictureBox1.Image = imgpreview;
                  
                    pictureBox1.Refresh();
                    string uniqueTag = "_" + DateTime.Now.ToUniversalTime().Ticks.ToString();
                    imgpreview.Save(Path.Combine("D:\\Test\\test", uniqueTag + ".png"));
                    memoryStream.Close();
                }
                catch {
    
                }
                finally
                {
                    memoryStream.Dispose();
                    GC.Collect();
                    GC.WaitForPendingFinalizers();   
                }
    


    NOTE . Running gc is a very time-consuming action, you should define the gc collect operation at the right time.

    Sincerely,
    neil hu


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    • Marked as answer by mgppgm Thursday, September 28, 2017 2:32 PM
    Thursday, September 28, 2017 6:57 AM
    Moderator

All replies

  • Maybe one of the byte arrays is not a valid image. For investigations, save the bytes (img.RecordImage) to some file using File.WriteAllBytes, then try opening this image file with some graphical editor or Visual Studio. This can be done in an additional try-catch.

    Thursday, September 28, 2017 5:23 AM
  • Hello mgppgm,

    >>Could you look at the progresschanged code (Where the error was thrown) and suggest maybe a better way of doing this, and why

    For your  code, the outofmemory exception occurs when create a lot of reference object result in memory leaking .

    using (MemoryStream ms = new MemoryStream(img.RecordImage))

    The following description from MSDN.

    when you use an IDisposable object, you should declare and instantiate it in a using statement. The using statement calls the Dispose method on the object in the correct way, and (when you use it as shown earlier) it also causes the object itself to go out of scope as soon as Dispose is called.

    In fact, the using statement are not much more than syntactic sugar for try-finally blocks.I have tested your code and the using statement seems that no object is released. I have optimized your code base on your description . the following code is my testing result.

    MemoryStream memoryStream;   
            private void backgroundWorker1_ProgressChanged(object sender, ProgressChangedEventArgs e)
            {
                string path = @"C:\Users\v-feih\Pictures\Saved Pictures\1.png";
                Byte[] s = GetPictureData(path);
                try
                {
                     memoryStream= new MemoryStream(s);
    
                    System.Drawing.Image imgpreview = System.Drawing.Image.FromStream(memoryStream);
                 
                    pictureBox1.Image = imgpreview;
                  
                    pictureBox1.Refresh();
                    string uniqueTag = "_" + DateTime.Now.ToUniversalTime().Ticks.ToString();
                    imgpreview.Save(Path.Combine("D:\\Test\\test", uniqueTag + ".png"));
                    memoryStream.Close();
                }
                catch {
    
                }
                finally
                {
                    memoryStream.Dispose();
                    GC.Collect();
                    GC.WaitForPendingFinalizers();   
                }
    


    NOTE . Running gc is a very time-consuming action, you should define the gc collect operation at the right time.

    Sincerely,
    neil hu


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    • Marked as answer by mgppgm Thursday, September 28, 2017 2:32 PM
    Thursday, September 28, 2017 6:57 AM
    Moderator
  • Thank you very much Neil. 

    I (half-way) understand what you are saying. My initial thought is calling WaitForPendingFinalizers is probably going to make the application sluggish, almost aggravating, when you have thousands of records. I wonder if, rather than waiting on each object destruction, I keep up with how many objects I am creating, then at a certain count sleep the application to allow GC to catch up? Is there a GC command to force a flush at a particular time?

    Also, I would want to only destroy the memory stream after the OS uses it to create the file. So forcing a flush of the objects may not be a good solution either.

    I just thought about using a persisted memory stream, but that would cause problems with clearing and sizing and read-only issues, etc, etc.

    Any further thoughts?

    Thanks


    • Edited by mgppgm Thursday, September 28, 2017 1:47 PM
    Thursday, September 28, 2017 1:38 PM
  • Viorel,

    Thank you for the idea! Just to be clear this application is in development, so I am using the same database for each test at the moment. I'm thinking that if a record image was corrupt the application would throw the error on every run. But I will keep your idea in mind for diagnoses.

    Thursday, September 28, 2017 1:43 PM
  • Neil,

    I just tried the application with your explicit GC commands on about 3000 records. The application ran slightly slower, but not noticeably, and the gui updating appeared much smoother(which may be an indicator of a slight reduction in speed). This may be the answer, going to run it some more.

    I'm marking this as the answer as it may take a while before being absolutely sure. The application raises the memory usage by 2%. but after the BGW finishes memory usage drops back down. I can live with this.

    Thank You!


    • Edited by mgppgm Thursday, September 28, 2017 2:32 PM
    Thursday, September 28, 2017 2:01 PM