how to effectively remove background of an image c#

Question

Hello i am a newbie in c# programming, using visual studio 2010. I am currently working on a project and i am stuck at the part where i must effectively remove the background of an image. I am working on images like flowers.

i am now currently using filters to filter out the background. however different image has different color values and i have no idea how to remove background for different images.

i use the follow codings:

ColorFiltering filter = new ColorFiltering();

      filter.Red = new IntRange(30, 100);
      filter.Green = new IntRange(0, 255);
      filter.Blue = new IntRange(0, 255);

      // apply the filter
      filter.ApplyInPlace(cropImage);

and every time i want to filter the image i have to change the filter values myself manually is there any way i can do it without doing it manually?

Answer 1

use this artical

http://www.oliveragustin.com/background-subtraction-in-c/

 

Answer 2

oh this also requires me to enter values. is there any way i can get the values just from the image as well?

Answer 3

Hi,

this may sound odd but the best way to remove a background of an non-indexed image is to create a new one and draw only the foreground.

MakeTransparent() will not work, because the numbers of backgroundColors will be too large.

A floodfill-like (magic wand like) algorithm will work good only on files with a great contrast of background and foreground - and you'll have to antialias the edges of the so created shape.

HSL-based removement of the background could work good, but it really depends on fine-tuning of the values/ranges that are processed.

There are other things one can think of, lile doing some morphological processing and darwing the found regions of the original image.

You can manually crop the image (lasso)...

It was good to know, how the  background of your images looks like, maybe we could then find a way (or at least a direction) to remove the background automatically.

Regards,

  Thorsten

Answer 4

Hi Thorsten, Thank you for the detail explanation. The images i am working on is on flowers where by i couldnt control what type of image as the image is being upload by the user.

example: http://yfrog.com/45dsc09109rj

 

 

Answer 5

Ok, I made some quick tests with your image. What I can say by now is that a floodfill like (or magic wand) gave the best results til now, always assumed that the process must be automaticalizable. A two/three step process with an increasing of contrast in first place was better than without. Best result til now was: 1. Apply a threshold or increase upper luminance and decrease lower luminance, 2. remove the inner part (blossom) by flood-replacing the colors with targetColor Transparent, 3.Merge the images in a reversed way, so that in the original image the part stays colored, which is transparent in the Mask (overlaying image). One thing to ask is the appearance of the peduncle of the flower. Keeping this in the resulting image will be very hard in an automated process, because the color-values of it are much nearer the background color-range than the colors of the blossom/bloom. If I find the time, I'll test some more and be back here to share the results. If you need help in writing the algorithms, ask, there are a lot of people around who like to help.

Regards,

  Thorsten

Answer 6

hi thorsten, yes i desperately needing help. having a hard time doing this and my project due this coming wed. :( thank you so much.

Answer 7

Hm, I don't think that it is possible to write a complete and reliable solution for this until wed, because the whole thing is really not a trivial task, because lots of parameters will be different in each new picture. If this really was easy then a hundrets of components were out that could do this. I think we first have to look for a way to get rid of the background for images that are like the one you uploaded, Background green/brown and blossom-part with a much stronger red(or blue) colorpart. If we get this to work reliably (which will not be easy), we have to think of how to automate this, means, how to parse the image to get the correct colorvalues and locations to start the processing (which could be even harder). Also, maybe the whole thing will not be able to be automated, this will experience then show...

Regards,

  Thorsten

Answer 8

Maybe there's another way. You say users will upload the images. So why not create a web-application/activeX-/silverlight-/or else-component that enables the user to do some image (pre-)processing "by hand". Provide some methods to erase the background and a tutorial on how to and let the user do this. Could that be a choice for you? The automated variant still could be developed and added in future versions.

Regards,

  Thorsten

Answer 9

Hi again,

I think, I found a way to get halfway good results in a way that can be automated - but only on images where the background is "darker" then the foreground.

1. strongly increase the contrast

2. either do it a second time of apply a threshold based on comparing the red(and blue) color vals

3. get the lightest pixel from/around center and do a floodfill based color replacement with a tolerance of around 127

4. merge this picture with the original by displaying the parts of the original that are transparent in the newly created image.

I don't know, if the image you posted is copyrighted in any way, so I can not upload the edited result-images from that picture, but I have a second one which is taken by me, so here's the original:

http://jdlqlw.blu.livefilestore.com/y1pJwU1BOh7oipIDiICvzS4pIkbr_NPYhR5euE0dFRVpL495ix-1Io7QTQu6PVPWuwFUz4FYslJHejm6D5BR58yIrFhJljbBiaf/DSCF0899.JPG?psid=1

2 times contrast

http://jdlqlw.blu.livefilestore.com/y1pTrxP1D7U35o76E_5QTVLfKzNxwgq8ru7Yv4vmYHcnTK4-zRnp4GC9CoYtZX2-_9Mt4sUfImdWITIO1MjjY1PrK3KAyWy2KiR/DSCF0899_c100x2_ff_r_127_invmerged.png?psid=1

1 time contrast and threshold

http://jdlqlw.blu.livefilestore.com/y1pSWSTag0je_zUq2cjCwLJ_ZLucCdIU2MhZikN-cNTZkRGrilJhazQiG3rT3efF2L9ntLPYikbPxVwy6KwkJapJMzFFvtHrp7E/DSCF0899_c100_th_r84_ff_r_127.png?psid=1

As you can see, there could be issues with connected-blossoms and perhaps some edge-processing like smoothing the outline could still be necessary.

I'll do some more tests over the weekend, if I find the time, but I dont't know, if a one-way-of-processing-strategy is possible...

I hope JohnWein has one of his brilliant ideas ...

Regards,

  Thorsten

Answer 10

Hello! thank you so much for your responds. btw, the image i show it as an example is taken by me. may i check with you what do you mean by applying a threshold based on comparing the red(and blue) color vals?

 

I did a program in c# for contrast but i am not sure if i did it correctly. the code as below:

 private void button1_Click(object sender, EventArgs e)
    {
      try
      {
        OpenFileDialog open = new OpenFileDialog();
        open.Filter = "Image Files(*.jpg; *.jpeg; *.gif; *.bmp)|*.jpg; *.jpeg; *.gif; *.bmp";
        if (open.ShowDialog() == DialogResult.OK)
        {
          pictureBox1.Image = new Bitmap(open.FileName);
          textBox1.Text = open.FileName.ToString();

          Bitmap finalImage = new Bitmap(pictureBox1.Image, 320, 240);
          pictureBox1.Image = finalImage;

          AForge.Imaging.Image.FormatImage( ref finalImage );

          AForge.Imaging.ImageStatistics statistics =
  new AForge.Imaging.ImageStatistics(finalImage);
          // get the red histogram

          AForge.Math.Histogram histogram = statistics.Red;
          // get the values

          double mean = histogram.Mean;   // mean red value

          double stddev = histogram.StdDev; // standard deviation of red values

          int median = histogram.Median; // median red value

          int min = histogram.Min;    // min red value

          int max = histogram.Max;    // max value

          // get 90% range around the median

          AForge.IntRange range = histogram.GetRange(0.9);

          //create levels filter

          AForge.Imaging.Filters.LevelsLinear filter =
                        new AForge.Imaging.Filters.LevelsLinear( );
        filter.InRed = statistics.Red.GetRange( 0.8 );
        filter.InGreen = statistics.Green.GetRange(0.8 );
        filter.InBlue = statistics.Blue.GetRange(0.8 );
        // apply the filter

          System.Drawing.Bitmap newImage = filter.Apply( finalImage);

          pictureBox2.Image = newImage;
            }
          }

      catch (Exception)
      {
        throw new ApplicationException("Failed loading image");
      }

     
    } 

pls kindly advise me thank you :)

Answer 11

i have another question, does your floodfill refering to this? http://www.codeproject.com/KB/GDI-plus/floodfillincsharp.aspx

Answer 12

Hi,

yes, floodfill I meant is like that, but I use an own written algorithm.

The suggestion with the threshold is just an idea I had. It makes a black and white image out of the picture, where black to white changes at the given threshold.

I recently had some problems with the processing-sequence I suggested, especially when the blossom has a dark inner part (which is quite often with flowers).

The base-pic:

http://jdlqlw.blu.livefilestore.com/y1poccL5Wy5nG9Cswmu-nmq5iRFvKZE67ZrZ4Q4k-PXXCAF8blmuTuxiaNEhlEKAWkbYs7oB08g54qXoTBC1NwpGl01dFLzp25Z/DSCF0874.JPG?psid=1

The result with the suggesteed sequence:

http://jdlqlw.blu.livefilestore.com/y1p5YW8ryrMdz-7xkhLaj5FQxZP0fXHd2n59djbJ592PTxSyWaeSqSWfyg5vPQTjgTPhlf3jtISdhwvVgkWt_UmDabPIUpVgLig/DSCF0874_1.png?psid=1

As you can see, there's a hole in the middle of the flower.

I'm trying to solve this by writing an algorithm that gets the outer-outline of the floodfilled range...

and am trying som other ways/sequences with this result:

http://public.blu.livefilestore.com/y1pvv9AK5mcQrNloFlzNAYnl0X4wTLSIV1kwXNmSXdu2FNdJsliLInnjMrx51-ebU45U3RMQL0_8WsBpvJ7NVNBFQ/c100x2_ff_r_127_invmrg_wid1x3_mrg_randzw_127sin5.png?psid=1

I'l keep on thinking and will be back,

Regards,

  Thorsten

Answer 13

Hi again,

here's some steps that you could do before beginning to write the code:

- take any good image editing program. There are some free/gpl-licenced like gimp or paint.net.

- try to create a mask that contains only the shape of the blossom of the flower *or* everything else

- try that on reducing colors, either by "reduce colors", or increasing contrast/color curves/threshold etc.

- try also getting the mask by using any tool that can be programmatically automated like a magic wand etc.

- log everything you do

- if you succeed on one picture, try do redo everything with exact the same parameters on other/different pictures.

- if you find any processing sequence that works for you, think of how to rebuild that with own code. Then we could help on writing the algorithms or giving links where they are free to use for copying/pasting/download.

Regards,

  Thorsten

Answer 14

hello, do you mean for example i make use of photoshop, and i remember the parameters and i try it on others images?
Firstly, i have to apply contrast, threshold, floodfill, merge image and so on?
cos i have no idea how am i gonna strt. very worried.

 

Answer 15

Yes, that's exactly what I mean. First you need to know, how your background-removement can be achieved reliably on different images with the same parameters, because only then it can be automated. Once that processing-sequence is found, coding can begin. Just try with an image-editing-program in every way you can imagine that this task could be successfully done, and *remember* each step you do.

Secondly, don't worry, this is a real complicated task IMHO, so it could take some time.

Btw, I forgot to ask: Why do you want to remove the background, what shall be done with these images?

Regards,

  Thorsten

Answer 16

okay, cos the project i am working on requires me to compare between 2 images(2 flowers) and i need to remove the background of the image in order for the comparison to be more accurate.

Answer 17

Hi,

here's some testcode for you (But this is really "testcode", use with care and be aware that there can be a lot of bugs in there):

http://cid-d5e5bd21dbf5e4e9.office.live.com/self.aspx/ng^_demo/ng^_Fragen/TestFloodfill.zip

I'm currently trying to make Mode 2 better ... with a floodfill-area-outline that again can be used for floodfill in the mask.

Regards,

  Thorsten

Answer 18

you are really a genius.

Answer 19

I have a question for you. does the "Go Modus" button refering to the increasing the contrast of an image?

Answer 20

Hi,

Modus 1 ("GoModus1") -sequence is:

increase the contrast

get the darkest pixel

get the mask by a floodfill-replace with Color.Transparent

merge image and mask

Modus 2:

two times contrast

get lightest pixel from middle-part of image

get the mask by a floodfill-replace with Color.Transparent in the blossom area

merge image and mask in "inverted" way (mask-opaque will be transparent in orig-picture, and mask_transp->opaque in orig)

But these are only prototypes, I will develop that further in future and do some code cleanup as well...

Regards,

  Thorsten

Answer 21

I added a third mode, which is a bit different, now we've got to test what mode whith what tolerance is best.

Regards,

  Thorsten

Answer 22

I am really grateful to you for your help. I am now trying to figure out the program. I have got another qns the get mask by a floodfill-replace with color.transparent refers to?

Answer 23

I have got another qns the get mask by a floodfill-replace with color.transparent refers to?

... the floodfill region and what's in it.

Yes it's fun to do something that's really interresting. Maybe, in future versions, this can produce reliable results (I hope - now, cause I didn't think it was possible, but it seems to be now).

I made some tests with combining mode1 and mode2:

http://cid-d5e5bd21dbf5e4e9.office.live.com/self.aspx/ng%5E_demo/ng%5E_Fragen/TestFloodfill.zip

Regards,

  Thorsten

Answer 24

I did some first code cleanup and will now change the combined method a bit and do further cleanup and error handling. So I think, by tomorrow the code will be readable.

Regards,

  Thorsten

Answer 25

Ok, I now changed the combined mode and did some tests with this image:

http://cid-d5e5bd21dbf5e4e9.office.live.com/self.aspx/ng^_demo/ng^_Fragen/Flowers/DSCF0874.JPG

because this image has a flower with light blossom but a dark part in the middle, which, I think, is quite often with flowers.

The application is still located here:

http://cid-d5e5bd21dbf5e4e9.office.live.com/self.aspx/ng^_demo/ng^_Fragen/TestFloodfill.zip

I think this combined mode could be the strategy-base for this task. Now we've got to increase the numbers ofg good results and do something for performance.

Regards,

  Thorsten

*##################################*

Edit: I changed the test for Point.IsEmpty to testing the x, and y-values of the point, to get a point(0, 0) as valid.

*##################################*

Answer 26

... maybe there's another option. Years ago, I wrote a simple color filter for the YCbCr-Colorspace, maybe with that the results can be better. I'll do some tests.

Regards,

  Thorsten

Answer 27

You say users will upload the images. So why not create a web-application/activeX-/silverlight-/or else-component that enables the user to do some image (pre-)processing "by hand". Provide some methods to erase the background and a tutorial on how to and let the user do this.

Hi, did you think about this as an option? The automated version will go on, but slowly. I recently added some HSL-based stuff, but to get it to work with a lot of different pictures will be very hard, I think.

Why not give the user some base methods which removes some of the background automatically and let the user simply "wipe" the rest-background away?

http://cid-d5e5bd21dbf5e4e9.office.live.com/self.aspx/ng^_demo/WipeAlpha.zip

Regards,

  Thorsten

Answer 28

oh hello, yes automation really cant work. i will consult my lecturer and tell you again. thank you so much :D

Answer 29

is it possible to explain how you get the darkest pixel from an image? and assign them to do a flood fill? i tried to figure out but to no avail.

 

 private Point GetDarkestPoint(string imageFile)
    {
      Bitmap bmp = new Bitmap(imageFile);
      BitmapData bmpData = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height), ImageLockMode.ReadOnly, bmp.PixelFormat);

      Point darkestPoint = new Point(0, 0);
      int darkestValue = 255 + 255 + 255;
      unsafe
      {
        byte* imgPtr = (byte*)(bmpData.Scan0);
        for
        (int i = 0; i < bmpData.Height; i++)
        {
          for
           (int j = 0; j < bmpData.Width; j++)
          {
            if
              (darkestValue > (*imgPtr) + *(imgPtr + 1) + *(imgPtr + 2))
            {
              darkestPoint = new Point(j, i);
            }
            imgPtr += 3;
          }
          imgPtr += bmpData.Stride - bmpData.Width * 3;
        }
      }

      return darkestPoint;
    }

Do i do it this way?

 

Answer 30

Hi,

don't forget (actually "never" forget) to unlock the bits again. Before you return something.

bmp.UnlockBits(bmpData);

return darkestPoint;

Lock it in 24bppRgb-mode, or get and set the size of one pixel to a variable when locking in "bmp.PixelFormat" -> this also could be 32bppArgb and the your offset has to be either 0 or stride - bmpWidth * 4 (which will be 0).  (I simply lock in 32bppArgb mode so a row will always be a multiple of 4)

The same is for incrementing the pointer. Either lock in 24bpp or increment by a variable "pixelSize".

Regards,

  Thorsten

Answer 31

If you mean assign a floodfill to my floodfill-methods then:

Get the Point,

get the color at this point (for start and tolerance),

fColor is the actual color in the image at this point,

the parameters are:

bitmap, x, y, tolerance, startColor, targetColor(color with which the fooding shall be made), MaxIterations

Color fColor = bmp.GetPixel(pt.X, pt.Y);

fipbmp.floodfill(bmp, pt.X, pt.Y, (int)this.numericUpDown1.Value, fColor, Color.Transparent, Int32.MaxValue);

The second overload takes additionally a bitarray which indicates for which color-components the calculation (tolerance) shall be made (bits are r,g,b,a) and the parameter f_addition is a bool to switch the calculationmode (add the values of the colors and then check for tolerance, or not).

btw, these are older methods, so variable-names might be not so "speaking", I will review the code sometimes ... at least I think so ...

Regards,

  Thorsten

Answer 32

Hello Thorsten Gudera

first of all ,thank you very much for the help you are providing
the application that you did is very helpful for me ,but not with all images
And , I am currently working on a project were i need to isolate the flower from the background or a leef from a tree because then I need to do some features extracting from the object that I isolated so i can compare it with some data in a database to get the flower information .
The comparing is going to be with a data like the one in the following url :
http://plants.usda.gov/java/largeImage?imageID=abma8_001_ahp.tif
the picture in the URL did not give me a good result

please ,can you help me with telling me what I need to do to get a good result most off the times ...
and when I am taking the flowers pictures should i have any consideration in taking it .
thanks alot...

Answer 33

Hi,

this is not easy to answer. What you could do is to read about Feature Extraction from images. But this is a large field. See: https://sites.google.com/site/cvonlinewiki/home/geometric-feature-extraction-methods

To get a good result with the particular image you linked above the code from me will not be sophisticated enough. What you could try to do is to create/determine a (set of) factors that indicate or discriminate the "inside/foreground" parts from the "outside/background" parts and feed these into for instance a chain code algorithm to get the outline of the "inside" parts. This outline then could be used as a starting point to exatract the inside parts, maybe easily by filling it with the texture of those image parts (filled path with a texturebrush in gdi+)

Those "factors" could also help in taking the pictures.

Regards,

  Thorsten

Answer 34

hi

thank you so much for responding .

since the project I'm working on almost reached it's deadline time ,so it has been changing on how we are going to do the comparing , and the first stage is:
- we assumed that the object is always in the center of the image whether it's a flower or a leaf
And now  we want to isolate this object that is located in the center of the image.
Can you tell me the mechanism of action to how to get to it and isolated from the background and if this can be achieved in your project, What are the required modifications to do that

thanks alot for your help .

Answer 35

...
And now  we want to isolate this object that is located in the center of the image.
Can you tell me the mechanism of action to how to get to it and isolated from the background and if this can be achieved in your project, What are the required modifications to do that
...

Hi,

hi, as stated above, there are many ways to do so and there's no general path to do, that always will give good results. I usually have the advantage, that - when extracting features form images - I can do this "by hand", so I can simply try and see whats best, but if you want to do this automatically, you might - in my opinion - hardly do that successfully without reading one or two books and study some math...(keywords: fearture extraction, but also neural networks, digital image processing in general, digital signal processing, image enhancement, homomorphic filtering and much more :-) )

What I can say is, a spatial region defined where the interresting part is located is good ([auto-]- correlation). A great contrast between background and foreground is better (floodfill-oriented, chain code oriented). Additionally a mainly one-colored flower is good also, but all that isnt given most of the time. Maybe a background-oriented consideration can help also. (If the background is always green/grown, maybe it could lead to better result when concentrating on removing the background parts instead of getting the foreground...)

Regards,

  Thorsten

Answer 36

Hi, your work is really good. I like the 'TestFloodFill' application. I wanted to recolor trees and bushes - i.e. a hue range of yellows and greens.  When  I run it on Windows 7 it works great.  When i run it on Windows XP 32 bit no changes are visible on the image.  Is your application only supposed to work on W7?  Or..?

Thanks for your time.

Best,

Andrew

Answer 37

Hi,

I never tested it on XP but it should work like on Vista/Win7. I'll do some XP tests and report the results...

Regards,

  Thorsten

Answer 38

Hi,

That's good of you.  

Thanks

Answer 39

Hi,

the problem is in the Form_LOad event. On xp systems the default picture (from resources) will be load in 24bpp colordepth. You can fix this by simply changing the line:

            Bitmap bmp = (Bitmap)global::TestFloodfill.Properties.Resources.DSCF0899.Clone();

to:

            Bitmap bmp = new Bitmap(global::TestFloodfill.Properties.Resources.DSCF0899);

so that the Form_Load eventhandler now looks as:

        private void Form1_Load(object sender, EventArgs e)
        {
            Bitmap bmp = new Bitmap(global::TestFloodfill.Properties.Resources.DSCF0899);
            _bmpBU = (Bitmap)bmp.Clone();
            _bmp = (Bitmap)bmp.Clone();
            this.pictureBox1.Image = (Bitmap)bmp.Clone();
            this.pictureBox1.AllowDrop = true;
        }

It seems that on xp bitmaps from resources arent automatically loaded as 32bit images, creating a new Instance of the Bitmap class (with this resources as orig-Image) will create a default bitmap with 32bpp color depth.

I will update the zip-file on skydrive in a couple of hours, when I find the time...

Regards,

  Thorsten

Answer 40

Hi,

I should have spotted/debugged that myself as I had a similar problem in 1 of my apps. Thanks for the quick response really appreciated it.

Best

Andrew