none
DirectShow and Kinect for Windows SDK RRS feed

  • Question

  • Hi,

    is it possible to write a DirectShow Filter which uses the Kinect for Windows SDK?

    Thanks!

    Friday, August 19, 2011 2:13 PM

Answers

  • I don't have sample code for exposing kinect as a video source, but it should not be too hard. You can open image stream using NuiImageStreamOpen without a callback event by specifying NULL for the 5th parameter (hNextFrameEvent), and then whenever you need a frame you call NuiImageStreamGetFrame. Frames will be given to you uncompressed, with VGA color frames having 4 bytes per pixel (X8R8G8B8 format) and depth image frames having 2 bytes per pixel, format as described in page 18 of programming guide (http://research.microsoft.com/en-us/um/redmond/projects/kinectsdk/docs/ProgrammingGuide_KinectSDK.pdf).

    If you want to use SDK functionality also within a filter, what do you want to use it for? As I mentioned above, the SDK can only perform skeleton tracking on the depth image coming from kinect device: there SDK is no mechanism to feed in an arbitrary depth image stream and perform skeleton joint tracking for it.

    Eddy


    I'm here to help
    Wednesday, August 31, 2011 5:41 PM

All replies

  • It should be possible by configuring the Kinect Audio DMO to be in filter mode: setting MFPKEY_WMAAECMA_DMO_SOURCE_MODE property to FALSE via  IPropertyStoreInterface. The MicArrayEchoCancellation sample (installed to C:\Users\Public\Documents\Microsoft Research KinectSDK Samples\Audio\MicArrayEchoCancellation\CPP with SDK installation) shows how to use Kinect DMO in source mode in a DirectShow context and MFAudioFilter (installed to C:\Users\Public\Documents\Microsoft Research KinectSDK Samples\Audio\MFAudioFilter\CPP) shows how to use Kinect DMO in filter mode, as a Media Foundation Transform. Between the two samples, hopefully it's easy to figure out how to configure it into a DirectShow filter.

    Edit: I just realized this question was posted to NUI forums rather than audio forums. Was this a question related to audio processing or video processing? If audio processing, I'll move this over to the right forum and my response still applies. If you meant video processing then the answer is no because the kinect video component works strictly as a data source, and does not have any mechanism for feeding data into it.

    Hope this helps,
    Eddy


    I'm here to help
    Saturday, August 20, 2011 12:07 AM
  • Hey Eddy,

    I am little confused. What if we want to use Kinect as a video source. Cannot one write a video capture filter?

    Thanks,

    Varun


    Varun Sharma
    Friday, August 26, 2011 7:08 PM
  • Sorry for the confusion, Varun. Yes, you should be able to wrap kinect video stream as a DirectShow source and then write a capture filter as a separate component.

    However, I understood the question being about having the Kinect video or audio functionality itself wrapped in a filter, e.g.: have a depth image stream source other than the kinect and then use the kinect video processing functionality to output skeleton joints. This second scenario is the one I referred to as not being possible with Kinect SDK Beta.

    Eddy


    I'm here to help
    Friday, August 26, 2011 11:02 PM
  • Thanks for the clarification and sorry for not asking the question precise enough.

    What would be a good startingpoint for writing such a filter?

    Thanks!

    Monday, August 29, 2011 9:16 AM
  • Alan, I'm still not 100% clear on whether you meant audio or video for the filter you want to do. Please clarify.

    Eddy


    I'm here to help
    Monday, August 29, 2011 8:59 PM
  • Sorry.

    It is about using the Kinect as a video source and process the video streams with the SDK within DirectShow filter(s). So that you can use the processed video inside an arbitrary application supporting normal webcams.


    Thanks!

    Tuesday, August 30, 2011 11:21 AM
  • I don't have sample code for exposing kinect as a video source, but it should not be too hard. You can open image stream using NuiImageStreamOpen without a callback event by specifying NULL for the 5th parameter (hNextFrameEvent), and then whenever you need a frame you call NuiImageStreamGetFrame. Frames will be given to you uncompressed, with VGA color frames having 4 bytes per pixel (X8R8G8B8 format) and depth image frames having 2 bytes per pixel, format as described in page 18 of programming guide (http://research.microsoft.com/en-us/um/redmond/projects/kinectsdk/docs/ProgrammingGuide_KinectSDK.pdf).

    If you want to use SDK functionality also within a filter, what do you want to use it for? As I mentioned above, the SDK can only perform skeleton tracking on the depth image coming from kinect device: there SDK is no mechanism to feed in an arbitrary depth image stream and perform skeleton joint tracking for it.

    Eddy


    I'm here to help
    Wednesday, August 31, 2011 5:41 PM