none
Is the liTimeStamp property exactly showing the Frame time of color image or Depth image? RRS feed

  • Question

  • Hi all,

    I am now going to take the human body out of color stream as a foreground, and it's fine when people doesn't move very fast in the camera. However, when player moves very fast, the depth image has an obvious delay to color image. 

     

    Hereby, I use the following code to match depth image and color image:

     

    if (index !=0)

    {

    time =  abs(temp1[index-1].liTimeStamp.QuadPart - (* pImageFrame).liTimeStamp.QuadPart);

    pointer = index-1;  

    index--;

    while(index !=0)

    {

    int time1 = abs(temp1[index].liTimeStamp.QuadPart - (* pImageFrame).liTimeStamp.QuadPart);

     

    if (time1<time)

    {

    pointer = index;

    time = time1;

    }

    index --;

    }

    }

     

    temp1 is an array which stores the color image in color stream, and when there is an depth image captured, I chose the closet color image to match depth image by comparing their liTimeStamp property. However the delay still exists.

     

    Has anyone met similar problems before?

     

    Best regards,

    Linsen

    Wednesday, September 14, 2011 9:47 AM

Answers

  • Frames matching each other exactly should have the exact same timestamp. However, you might still see a delay if some image frames (depth frames, most likely) were dropped by the runtime before runtime had a chance to deliver them to you. This is possible due to delays caused by operating system multitasking and/or heavy processing performed by your application (potentially, depending on your application) on each frame.

    Are you using the event handling model to get depth and RGB frames, or polling model (see programming guide http://research.microsoft.com/en-us/um/redmond/projects/kinectsdk/docs/ProgrammingGuide_KinectSDK.pdf, page 19) to synchronize between depth and RGB color streams? The polling model is more complicated to code but produces better results in current SDK. You can see an example of synchronizing these two streams in forum thread http://social.msdn.microsoft.com/Forums/en-US/kinectsdknuiapi/thread/c5c00b90-28d7-49f5-b451-d79609604ec0.

    Sample code in referenced thread will still not result in perfect synchronization since there is still a (smaller) chance of dropped frames, but it should be about as close as you can get them with Kinect SDK beta.

    Eddy


    I'm here to help
    Thursday, September 15, 2011 8:45 PM

All replies

  • Frames matching each other exactly should have the exact same timestamp. However, you might still see a delay if some image frames (depth frames, most likely) were dropped by the runtime before runtime had a chance to deliver them to you. This is possible due to delays caused by operating system multitasking and/or heavy processing performed by your application (potentially, depending on your application) on each frame.

    Are you using the event handling model to get depth and RGB frames, or polling model (see programming guide http://research.microsoft.com/en-us/um/redmond/projects/kinectsdk/docs/ProgrammingGuide_KinectSDK.pdf, page 19) to synchronize between depth and RGB color streams? The polling model is more complicated to code but produces better results in current SDK. You can see an example of synchronizing these two streams in forum thread http://social.msdn.microsoft.com/Forums/en-US/kinectsdknuiapi/thread/c5c00b90-28d7-49f5-b451-d79609604ec0.

    Sample code in referenced thread will still not result in perfect synchronization since there is still a (smaller) chance of dropped frames, but it should be about as close as you can get them with Kinect SDK beta.

    Eddy


    I'm here to help
    Thursday, September 15, 2011 8:45 PM
  • Hi Eddy,

     

    Thanks for your reply!

     

    In referenced thread, you use a FrameNumber property to match Depth and Color image. Is this property the time when frame is captured? Or the number when frame is captured?

    Best regards,

    linsen

    Friday, September 16, 2011 8:02 AM
  • that's the frame number. You could also match on time stamp. I'm aware that in threads from a month ago or longer I was recommending to match frames using timestamps rather than frame numbers, but it turns out that if you write enough code to store old RGB image frames and later do a lookup (as in linked sample), it works just as well using frame numbers as using time stamps.

    Eddy


    I'm here to help
    Friday, September 16, 2011 9:03 AM