The following forum(s) are migrating to a new home on Microsoft Q&A (Preview): Developing Universal Windows apps!

Ask new questions on Microsoft Q&A (Preview).
Interact with existing posts until December 13, 2019, after which content will be closed to all new and existing posts.

Learn More

 none
Video Decoding of IMFSample in Direct3D 11, UWP, (send IMFSample to ID3D11VideoContext), C++ RRS feed

  • Question

  • Hi everyone, so I'm working on video streaming plugin and so far so good. I have CLSID_CMSH264EncoderMFT which is gives me IMFSample that i send via network and process it with CLSID_CMSH264DecoderMFT, like:

    pTransform->ProcessInput(inOutsteamID, videoSample, 0);

    Everything is perfect, but now i wanna add GPU hardware acceleration, so i follow the instruction from microsoft: "supporting-direct3d-11-video-decoding-in-media-foundation". Its not perfect but everything more or less clear, until i met this: 

    1. Call ID3D11VideoContext::SubmitDecoderBuffer. This method instructs the decoder device to perform the decoding operations on the frame.

    My question is, does this function want from me? As far as i understand i need to fill this buffers: 

    D3D11_VIDEO_DECODER_BUFFER_PICTURE_PARAMETERS
    D3D11_VIDEO_DECODER_BUFFER_INVERSE_QUANTIZATION_MATRIX
    D3D11_VIDEO_DECODER_BUFFER_BITSTREAM
    D3D11_VIDEO_DECODER_BUFFER_SLICE_CONTROL

    except i have only IMFSample, what should i do? Should i decode somehow every sample to get h.264 sps/pps fields and fill giant struct DXVA_PicParams_H264? is it the only way of doing that? If so, how to get all of this information from single sample? Or should i use:

    pMFTOutputMediaType->GetBlob(MF_MT_MPEG_SEQUENCE_HEADER, &blob.front(), sequenceHeaderDataSize, NULL);

    and get sps/pps from that blob? And what is SLICE CONTROL, i don't have any slice info, is it somewhere in sample? Is there any chance to have sample, tutorial of how to send IMFSample to the GPU and decode it? 

    If you knew any of that or have some links to tutorials, instructions of how to do that, it will helps me a lot! Also, would be cool if there is any microsoft tutorial for that



    • Edited by 4ipideil Tuesday, July 16, 2019 8:29 AM
    Monday, July 15, 2019 4:31 PM

Answers

  • Hi,

    I've got the response from the team. Currently there is no such function in the available SDKs.

    Best regards,

    Roy


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    • Marked as answer by 4ipideil Wednesday, July 31, 2019 9:47 AM
    Tuesday, July 30, 2019 8:39 AM
    Moderator

All replies

  • Hi,

    I'll need to ask other engineers about this question. There might be some time delay. Thank you for your patience.

    Best regards,

    Roy


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Tuesday, July 16, 2019 7:55 AM
    Moderator
  • okay i was able to read the sample and get the byte array for sps and pps (its in the MF_MT_MPEG_SEQUENCE_HEADER in encoder output IMFMediaType and in the first sample that i send). Also i can recognize the slices inside IMFSample, but what should i do with all of them? Somehow manually decode them and fill DXVA_Slice_H264_Short and DXVA_PicParams_H264? Is there any tools for that? also i need to push whole IMFSample buffer into D3D11_VIDEO_DECODER_BUFFER_BITSTREAM or a part of it?
    Tuesday, July 16, 2019 5:21 PM
  • okay i found how to get everything, but my last problem is about: 

    How to get the size of each NAL unit? i knew that IMFSinkWriter write the media file with the size in NAL headers, but how to make that size to be set in IMFTransform::ProcessOutput

    Is there a good way of doing that, for now i can just decode the NAL units by the code from the book "Coding of moving video H.260–H.279", but im sure its not the way of doing that, at some point IMFTransform should know about all of this NAL units as its the one who pack them together, so how can i get them?

    Wednesday, July 17, 2019 6:40 PM
  • Hi,

    I'm waiting for the response from another engineer. I'll come back as soon as I got response.

    Best regards,

    Roy


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Friday, July 19, 2019 9:05 AM
    Moderator
  • Hi,

    I've got the response from the team. Currently there is no such function in the available SDKs.

    Best regards,

    Roy


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    • Marked as answer by 4ipideil Wednesday, July 31, 2019 9:47 AM
    Tuesday, July 30, 2019 8:39 AM
    Moderator