none
Duplicate timestamps and irregular frame cadence RRS feed

  • Question

  • I'm seeing color (and sometimes depth) frames with duplicate timestamps. I'm also seeing dropped framess and irregular frame frequencies.  Here is an example:

    12540.22ms: depth packet 76, timestamp = 8752
    12570.10ms: depth packet 77, timestamp = 8783
    12584.16ms: rgb packet 74, timestamp = 8752
    12587.99ms: rgb packet 75, timestamp = 8799
    12613.31ms: rgb packet 76, timestamp = 8799
    12616.06ms: rgb packet 77, timestamp = 8830
    12631.83ms: depth packet 78, timestamp = 8846
    12644.89ms: rgb packet 78, timestamp = 8861
    12663.83ms: depth packet 79, timestamp = 8877
    12676.30ms: rgb packet 79, timestamp = 8892
    12710.10ms: depth packet 80, timestamp = 8908
    12734.52ms: depth packet 81, timestamp = 8955
    12738.81ms: rgb packet 80, timestamp = 8955
    12743.09ms: rgb packet 81, timestamp = 8955
    12785.44ms: depth packet 82, timestamp = 8986
    12789.40ms: rgb packet 82, timestamp = 9002
    12803.45ms: depth packet 83, timestamp = 9017
    12808.47ms: rgb packet 83, timestamp = 9017

    Above, rgb packets 74 through 77 arrive in 31.9 milliseconds and contain one duplicate timestamp.  There is a depth frame missing from that time range.
    Rgb packets 80 and 81 also have duplicate timestamps and arrive within 4.28ms of one another.
    This is quite annoying for apps that depend on timestamps... Seems like there should never be frames delivered with the same timestamps...  Are frames with duplicate timestamps guaranteed to be identical?  Can I just ignore them, or do I need to restamp them with a real time?

     

    Tuesday, February 14, 2012 1:50 AM

All replies

  • Another example, with both duplicate depth and rgb timestamps:
    10652.30ms: rgb packet 73, timestamp = 6910
    10697.47ms: depth packet 74, timestamp = 6957
    10712.03ms: depth packet 75, timestamp = 6957
    10726.41ms: depth packet 76, timestamp = 6973
    10730.02ms: rgb packet 74, timestamp = 6910
    The color data in the two frames is different.
    This seems bad to me...
    Tuesday, February 14, 2012 2:34 AM
  • Taking a look at the data above, I see:

    12540.22ms: depth packet 76, timestamp = 8752
    12570.10ms: depth packet 77, timestamp = 8783
    12584.16ms: rgb packet 74, timestamp = 8752
    12587.99ms: rgb packet 75, timestamp = 8799
    12613.31ms: rgb packet 76, timestamp = 8799
    12616.06ms: rgb packet 77, timestamp = 8830
    12631.83ms: depth packet 78, timestamp = 8846
    12644.89ms: rgb packet 78, timestamp = 8861

    RGB packets 74-77 originated over 78ms (which is less than the ~100ms expected), but there are two depth frames from that time period as well.

    >Can I just ignore them, or do I need to restamp them with a real time?

    You should not do that, since that would very likely be incorrect. 

    Ultimately, yes, this can happen - we are, at times, at the mercy of the thread scheduler, CPU spikes and, during startup of a sensor, initialization logic that can "peg" the Kinect itself.  What we have seen, generally, is that if an application is taking < 33ms to process each set of frames (that is, if it's "keeping up"), the frames are delivered very smoothly.  If the application is on the cusp of taking > 33ms for each set of frames, or if your system is overloaded for some other reason, or during sensor (re)init, frames can "bunch up" in the system. 

    What we've done is to ensure that (a) the timestamp is as accurate as we can currently make it (we're working on improving this) and (b) successive frames will always have a later or equal timestamp and (c) frame numbers will increase monotonically.  So, as you can see with the duplicate timestamp at 8799, the frame number is a "tie breaker" (or you can always sort by frame number, if you wish).

    We'd love to understand more about the scenario when you're hitting this - perhaps we can offer a suggestion, or perhaps it may provide insight to where our framework is vulnerable to this issue.


    -Adam Smith [MSFT]

    Tuesday, February 14, 2012 5:42 AM