none
Kinect v2 color camera's real resolusion is 1920x1080 or 960x1080? [Fixed in 1410] RRS feed

  • Question

  • Hi all,

    I found the image captured by Kinect v2 color camera are all dual-pixel pairs.

    I'm just wondering: it's extended from 960x1080 to 1920x1080 or there is something wrong with my hardware or settings?

    Anyone know this?


    Regards, Nighting Liu


    Monday, September 29, 2014 3:11 PM

Answers

All replies

  • KinectSensor.ColorSource.FrameDescription will provide your the camera values for the particular source. Color is 1920x1080 @ 30fps if full-light or 15fps in low light.

    Are you using the API's to get the value or did you get it from somewhere else?


    Carmine Sirignano - MSFT

    Monday, September 29, 2014 5:48 PM
  • Hi Carmine,

    The following is the image I directly copied from SDK Color-Basic demo, you will see: vertical pixels seem correct, but these lateral pixels seem are same two pixels as pair.

    Can you capture a image by Kinect with original size and send it to (nighting@outlook.com) to let me know if there is something wrong with my hardware? Thank you very much!


    Regards, Nighting Liu

    Tuesday, September 30, 2014 1:30 PM
  • That's interesting - looking at the data that is copied to an array using the CopyConvertedFrameDataToArray() - it seems the horizontal pixels are the same in 2-pixel pairs.

    Im not sure if that's the conversion from the Yuy2 format to the BGRA format that's causing it - but that is an interesting find

                           

    byte[] colorData = newbyte[colorFrame.ColorFrameSource.FrameDescription.LengthInPixels * colorFrame.CreateFrameDescription(ColorImageFormat.Bgra).BytesPerPixel];

                            colorFrame.CopyConvertedFrameDataToArray(colorData,

    ColorImageFormat.Bgra);

    Tuesday, September 30, 2014 2:32 PM
  • Interesting article:  https://support.microsoft.com/kb/294880

    The variations in the different YUV samples are based on how data is sampled, both in the horizontal and vertical directions.  The horizontal subsampling interval describes how frequently across a line that a sample of that component is taken, and the vertical interval describes on which lines samples are taken. For example, if the format has a horizontal subsampling period of 2 for both the U and V components, it indicates that U and V samples are taken for every second pixel across a line. If the vertical subsampling period is 1, it indicates that U and V samples are taken on each line of the image.

    Tuesday, September 30, 2014 3:43 PM
  • I'm definitely getting the same result (a 960x1080 image stretched out to 1920x1080). YUV sampling would not explain that result.

    http://en.wikipedia.org/wiki/Chroma_subsampling

    In 4:2:2 color space the hue and saturation values a sampled in horizontal pixel pairs, but luminance is still sampled for every pixel. This should still create a difference in each pixel, but here each pair is identical.

    It's not the worst thing in the world if the HD camera is subsampled horizontally for bandwidth/performance reasons. But it would be nice to have official confirmation so we know what we're actually working with.

    Tuesday, September 30, 2014 9:24 PM
  • Here is an example with the source on the left and the luminance and color channels separated to the center and right respectively.

    You can see the color is subsampled in 4 pixel 2x2 blocks. It's possible that the image coming out of the Kinect is a 960x1080 image in a 4:2:0 color space which is then stretched out to 1920x1080.

    If I'm not mistaken that makes the color image coming out of the Kinect a 2:2:0 color space, which is not an actual color space. I don't have any samples from the earlier version of the SDK so this could be a bug in the most recent version update.

    Tuesday, September 30, 2014 10:12 PM
  • Actually I just found a screenshot from the Dev Preview back in June. No pixel pairs, proper 4:2:0 colorspace.

    “This is preliminary software and/or hardware and APIs are preliminary and subject to change”

    Tuesday, September 30, 2014 10:21 PM
  • Hi Microsoft, can you explain this? From the following posts, it seems not a problem only happened in my situation.

    I just hope it will be bug in the SDK and will be fixed soon, thank you!


    Regards, Nighting Liu

    Wednesday, October 1, 2014 8:06 AM
  • Hi Microsoft,

    I need your feedback. Everyone writes bug, it really doesn't matter. I just want to know if you will fix this issue and if you will, when?

    Thank you very much!


    Regards, Nighting Liu

    Thursday, October 2, 2014 1:27 PM
  • Still no response to whether or not this is a bug or intentional.

    In the mean time you could interpolate the missing pixels using something like this (not tested). It should reduce the blockiness a little bit. I'm sure it could be better.

    for(int y = 0; y < 1080; y++)
    {
    	for(int x = 0; x < 1917; x+=2)
    	{
    		int index = (y * 1920 + x) * 4;
    
    		_ColorData[index + 4] = (byte)( ( _ColorData[index] + _ColorData[index + 8] ) / 2 );
    
    		_ColorData[index + 5] = (byte)( ( _ColorData[index + 1] + _ColorData[index + 9] ) / 2 );
    
    		_ColorData[index + 6] = (byte)( ( _ColorData[index + 2] + _ColorData[index + 10] ) / 2 );
    
    	}
    	int lastPixel = (y * 1920 + 1917) * 4;
    
    	_ColorData[lastPixel + 4] = (byte)( ( _ColorData[lastPixel] + _ColorData[lastPixel + 8] ) / 2 );
    
    	_ColorData[lastPixel + 5] = (byte)( ( _ColorData[lastPixel + 1] + _ColorData[lastPixel + 9] ) / 2 );
    
    	_ColorData[lastPixel + 6] = (byte)( ( _ColorData[lastPixel + 2] + _ColorData[lastPixel + 10] ) / 2 );
    
    }

    Please remember that this does not add any detail into the image, it simply interpolates the missing data. Microsoft needs to let us know if this is intentional or a bug that is going to be changed.

    Thursday, October 2, 2014 5:41 PM
  • Thank you for the report, we are investigating.

    Chris White _MSFT_

    Monday, October 6, 2014 6:10 PM
  • Thank you so much.

    In fact, I do need that full 1920x1080 resolution because I'm working on a facial recognition project.


    Regards, Nighting Liu

    Tuesday, October 7, 2014 3:29 AM
  • Hi Chris,

    I'm very glad to hear you are investigating, not "By Design".

    Also, I know you are the PM of Kinect, I do believe the facial tracking will be a killer application, if we get enough accurcy, even it can replace the mouse. But to achieve this, MS still needs to improve the resolution of color camera and TOF camera.

    Hope you can consider this!

    Best Regards.


    Regards, Nighting Liu

    Wednesday, October 8, 2014 2:24 AM
  • Looks interesting but i dont have these problems... Im using the 1408 sdk and the c++ interface.

    Wednesday, October 8, 2014 1:37 PM
  • Very glad to know there is a system works normally, but I still cannot figure out what's wrong with my system.

    Also many people can repro my problem.


    Regards, Nighting Liu

    Wednesday, October 8, 2014 3:53 PM
  • For facial recognition, we have found that working off of the lighting independent IR feed is *significantly* better than working off of the color camera. I would recommend investigating that path.


    Chris White _MSFT_

    Wednesday, October 8, 2014 6:28 PM
  • Yes, your colleage told me the same, but I just found that IR feed's resolution is very low, if I want to recognize people from 1500~2000, that resolution is far from enough. What do you think?

    BTW, do you mind give me your email address, I can share some of my findings on this. Maybe we can make Kinect better together, considering I ever been a MSer:)


    Regards, Nighting Liu

    Thursday, October 9, 2014 1:43 AM
  • Well, we do face unlock on xbox at 4+ meters using the IR sensor... As always there are tradeoffs inherent in any solution.

    Using the color feed introduces massive variation based on ambient lighting conditions... In our experience, this variation was *much* harder to overcome than the lower resolution associated with the IR camera. 

    If you do use IR (or investigate using IR), be sure to use the Long Exposure IR feed.  This will vastly lower the amount of noise in the signal, which will help with the analysis task.

    My guess is that you could do an even better job by combining the two (though each thing would have their own challenges. :)

    Happy to continue the conversation off forum. :) Chris.White@Microsoft.com


    Chris White _MSFT_

    Thursday, October 9, 2014 5:39 AM
  • Hi Chris,

    Any update on this issue?


    Regards, Nighting Liu

    Monday, October 13, 2014 5:28 PM
  • Thanks for Microsoft, you fixed this issue in version 1410.

    Regards, Nighting Liu

    • Marked as answer by Nighting Liu Monday, October 27, 2014 3:37 AM
    Monday, October 27, 2014 3:37 AM
  • Sorry for Asking my question on this issue but I did't find any better site and issue to ask the question. This is more relevant according to my scope. 

    Basically I was trying to reduce the pixel of Kinect from 1920*1080 to 640*480 / 320*240 because for my scope this resolution is too high. I have tried it in many ways but remains fails. you people are really expert if you have any idea to solve this problem get solve? 

    Any help will appreciation from me.

    Many Thanks. 


    • Edited by MOeez Raja Sunday, March 5, 2017 7:42 AM
    Sunday, March 5, 2017 7:38 AM