Improve performance of live depth view + colored bodies RRS feed

  • Question

  • I'm having problems getting a live view of the depth map + colored bodies performant on MonoGame 3.2.

    The approach I've takes is as follows (using reactive extensions but that should not matter):

    • MultiSourceFrameReader
    • Acquire depth frame + body frame
    • Pass onto an event loop
    • Convert the depth array + the body index array into a color array (on the CPU)
    • Copy the color array into a texture

    This works quite okay on my development PC which is a 3rd generation i7 with an NVidia GTX 660. Unfortunately this slows down the entire system on our target machine, a 4th generation i5 with Intel HD Graphics 4600 (I think..). The obvious solution to drop frames, which I'm not too happy about.

    Any ideas how to make the live view more fluent?

    Thursday, August 7, 2014 8:43 AM

All replies

  • There are known perf issues with MultiSourceFrameReader(MSFR) which are being worked on, but you should be able to get something similar to the WPF sample. Does MonoGame use the .Net framework or is there a wrapper around the COM apis? This is going to be dependent on what the system is capable of handling (see basics are you getting 30fps) and add on top of that, the requirements for your app.

    My recommendation for a game or any platform that is frame rate dependent, would be to use the polling model instead of MSFR. On setup create the readers and on each update:

    • check for the color frame(acquire latest), if none, exit and wait for next frame
    • acquire depth/body, if none wait for next frame
    • copy the depth/body data, and then copy the color frame
    • release all of the acquired Kinect frame objects
    • process the copied data

    Carmine Sirignano - MSFT

    Tuesday, August 12, 2014 6:05 PM
  • Interesting suggestion, I'll wait for the next SDK update before I decide what to do. The reason we currently use the MultiSourceFrameReader is that we use Reactive Extensions all over the place for gestures and body tracking based on playing fields.

    Wednesday, August 13, 2014 8:04 AM
  • I tried the polling mode, but AcquireLatestFrame seems to be a resource hog itself. For now I ended up listening for FrameArrived to spin up a Task that acquires the frame and processes it.
    Monday, August 18, 2014 1:14 PM