none
problem while convert Kinect depth frame to EmguCV RRS feed

  • Question

  • Hi community,

    I m doing a Kinect SDK project in WPF.I got a problem when i use openCV to do some image processing to Kinect depth frame, When i convert the bitmap to Image<Bgr, Byte>, the image i got after convert is very small and even duplicated, here is the picture and code.

    Can anyone help me and sorry for my English

     

     

       Kinect depth frame        OpenCv frame after convert

     

     

     

    PlanarImage Image = e.ImageFrame.Image;

    Bitmap bmap = PImageToBitmap(Image);


    Image<Bgr, Byte> currentFrame = new Image<Bgr, byte>(bmap);

    image1.Source = ToBitmapSource(currentFrame );




    • Edited by wizardliu Thursday, August 18, 2011 2:31 AM
    Tuesday, August 16, 2011 11:08 AM

Answers

  • The pixel format in the PImageToBitmap method is wrong, for the depth image it should be Format16bppGrayScale.

    Keep in mind that only the 12 lsb of the 16 bits are used, so trying to display it directly will result in black image. After youre done with the image processing, you should shift the data to the right or left by 4, depending if you want to display it as a 8bpp or 16 bpp gray scale image.


    • Proposed as answer by Eddy Escardo-Raffo [MSFT] Wednesday, August 17, 2011 6:59 PM
    • Marked as answer by wizardliu Thursday, August 18, 2011 1:11 AM
    • Unmarked as answer by wizardliu Thursday, August 18, 2011 2:23 AM
    • Marked as answer by wizardliu Thursday, August 18, 2011 2:23 AM
    Wednesday, August 17, 2011 11:35 AM

All replies

  • I assume that the code that you pasted is the code that doesn't work? Could you also paste a sample of the code that does work and confirm for me that the code that doesn't work also corresponds to depth stream rather than VGA color image stream?

    Also, what do these functions do (if you have code implementations for them, that's best, if you have a reference to documentation, that could be ok):
    - PImageToBitmap
    - Image class constructor
    - ToBitmapSource

    Also, for anything that is OpenCV-specific, have you asked the OpenCV group at http://tech.groups.yahoo.com/group/OpenCV/?

    If you describe the format of the image coming from Kinect into their API, they will probably be a better resource for telling you what to change in your code in order to use their API to do what you want it to do.

    Eddy


    I'm here to help
    Tuesday, August 16, 2011 11:26 PM
  • Thanks Eddy for your reply,

    The code is working, just that after i transferred the Kinect depth frame to EmguCV Image<Bgr, Byte>,the image became quarter size of the depth frame which that Kinect SDK provided and even duplicated(see the picture show).

     

    - PImageToBitmap

    //is to convert planar image to bitmap so that EmguCV can feed the data in


            public Bitmap PImageToBitmap(PlanarImage PImage)
            {

                Bitmap bmap = new Bitmap(PImage.Width, PImage.Height, PixelFormat.Format32bppRgb);
                BitmapData bmapdata =    bmap.LockBits(new Rectangle(0, 0, PImage.Width, PImage.Height), ImageLockMode.WriteOnly, bmap.PixelFormat);

                IntPtr ptr = bmapdata.Scan0;

                Marshal.Copy(PImage.Bits, 0, ptr, PImage.Width * PImage.BytesPerPixel * PImage.Height);

                bmap.UnlockBits(bmapdata);

                return bmap;

            }


    - Image class constructor

    A image type of EmguCV


    - ToBitmapSource

      public static BitmapSource ToBitmapSource(IImage image)
            {
                using (System.Drawing.Bitmap source = image.Bitmap)
                {
                    IntPtr ptr = source.GetHbitmap(); //obtain the Hbitmap

                    BitmapSource bs = System.Windows.Interop.Imaging.CreateBitmapSourceFromHBitmap(
                        ptr,
                        IntPtr.Zero,
                        Int32Rect.Empty,
                        System.Windows.Media.Imaging.BitmapSizeOptions.FromEmptyOptions());

                    DeleteObject(ptr); //release the HBitmap
                    return bs;
                }
            }

     

     

    Thanks for your advice,I already posted a question in EmguCV forum , just want to ask if anybody has used emguCV to access the kinect depth frame to do some further image process and is it a common problem or something wrong with  my code?

    Wednesday, August 17, 2011 3:10 AM
  • The pixel format in the PImageToBitmap method is wrong, for the depth image it should be Format16bppGrayScale.

    Keep in mind that only the 12 lsb of the 16 bits are used, so trying to display it directly will result in black image. After youre done with the image processing, you should shift the data to the right or left by 4, depending if you want to display it as a 8bpp or 16 bpp gray scale image.


    • Proposed as answer by Eddy Escardo-Raffo [MSFT] Wednesday, August 17, 2011 6:59 PM
    • Marked as answer by wizardliu Thursday, August 18, 2011 1:11 AM
    • Unmarked as answer by wizardliu Thursday, August 18, 2011 2:23 AM
    • Marked as answer by wizardliu Thursday, August 18, 2011 2:23 AM
    Wednesday, August 17, 2011 11:35 AM
  • Thanks Valrakk,

    Really appreciate for your reply,

    I got an error of "Parameter is not valid" after i changed the format.

    I am just a kinect starter and not good enough in C# also, can you show me some details in bit shifting as well?

     Thanks again for your help :)

    Thursday, August 18, 2011 2:30 AM
  • Try PixelFormat.Format16bppRgb565 and see if that one is supported. 16-bit grayscale isn't as widely supported as 16-bit color. You won't get the right color mapping by using Format16bppRgb565, but at least you should get correct image dimensions. If you do get correct image dimensions by using this false color format, then you can move on to the next problem, which is displaying the right colors you want to show on screen.

    Eddy


    I'm here to help
    Thursday, August 18, 2011 2:44 AM
  • Thanks Eddy~ At least i got my image processed now and it just a little bit lagged .

    I am actually using kinect SDK to get the depth data of human and use openCV to do some further bolb detction for objects.

    Thanks for your help for so many days, and thanks for your patience !

     

    Thursday, August 18, 2011 8:26 AM