How to send sequence header or extended format data to MFT decoder? RRS feed

  • Question

  • I am having trouble configuring MFT decoders. I think the reason is that I have not been able to figure out how to configure or send "extradata" (sequence header, extended format data) to the decoders. For Directshow, this is usually just added after the FORMAT_VideoInfo or FORMAT_VIDEOINFO2 data and setting the cbSequenceHeader to indicate the size of the sequence header data.

    For example, the documentation for the MPEG4 Part 2 Video decoder says the decoder accepts the following formats: VIDEOINFOHEADER, VIDEOINFOHEADER2 etc, but not how to configure the coder with any of these..

    The MSDN documentation for Configuring MFT Inputs and Inputs, says " Without the extended format data, you cannot decode the compressed content", but again, not how to send the ext. format data to the decoder.

    Any advice ?

    Monday, April 15, 2013 5:09 PM

All replies

  • I think I found part of the answer. 

    pInputMediaType->SetBlob(MF_MT_USER_DATA, pData, cbData)

    When I do this for the WMV3 decoder, and set the Subtype to MFVideoFormat_WMV3, at least the decoder's SetInputType succeeds.

    I am still not getting decoded data out. I keep sending data to ProcessInput, with no errors, but ProcessOutput just reports more input data needed, so something is still missing, either from the initialization or the media samples sent to the decoder.

    For the MPEG4 decoder, it's not clear if MF_MT_USER_DATA or MF_MT_MPEG_SEQUENCE_HEADER should be used, if either. I have tried both, but same problem: sending compressed data in, nothing ever comes back out.

    Monday, April 15, 2013 6:02 PM
  • Using the MF Blog sample MFMediaPropDump, I see my test file's properties as:

            MF_MT_MAJOR_TYPE: MFMediaType_Video
            MF_MT_SUBTYPE: MFVideoFormat_WMV3
            MF_MT_FRAME_SIZE: 640x480
                0x4D, 0x79, 0x4A, 0x11, 0x00
            MF_MT_AVG_BITRATE: 1500000
            MF_MT_FRAME_RATE: 25.00000fps
            MF_MT_PIXEL_ASPECT_RATIO: 1:1
            MF_MT_INTERLACE_MODE: MFVideoInterlace_Progressive

    I am setting the input type as follows:

            CHECK_HR( hr = MFCreateMediaType(&pInputMediaType) );
            CHECK_HR( hr = pInputMediaType->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video) );

            SubType = MFVideoFormat_WMV3;
            CHECK_HR( hr = pInputMediaType->SetGUID(MF_MT_SUBTYPE, SubType) );

            CHECK_HR( hr = pInputMediaType->SetBlob(MF_MT_USER_DATA, m_cd.extradata, m_cd.extradata_size) );


    So now I have setup the decoder with the correct parameters - yet it refuses to decode my samples, or rather ignores them, as there are no errors passing the samples in. It just won't offer any decoded data.

    This is mildly frustrating :) The only MFT decoder I have been able to get to work is the H.264 decoder. All other decoders refuse to process data, and offers no clues as to what's going on. 

    Monday, April 15, 2013 9:43 PM
  • Continuing the conversation with myself. I finally found the solution.

    Not giving up quite yet – one last idea. If I use an MF source reader to read samples from the same file that I am reading from another source reader, I see the raw data buffer is exactly the same, but for these samples, data comes back out from the decoder! 

    So I just needed to figure out what magic is missing from the IMFSample I am creating, compared to the IMFSample that comes out from the MF source reader. Sounds simple, but these objects are rather complex and can only be parsed programmatically. Fortunately there is a method to retrieve items by index without knowing their GUID name – I did this and for each numerical GUID found the corresponding name in the right SDK header file. It turns out the only missing magic item was a property called MFSampleExtension_CleanPoint, which indicates a key frame (this appears to be needed for the first frame, even though the information is also in the bitstream – the decoder cannot seem to figure this out on its own). 

    The MFSampleExtension_Discontinuity flag which I had already been setting was not enough.

    So now I can decode WMV3 and MPEG-4 with MFT after all.

    // MFSampleExtension_CleanPoint {9cdf01d8-a0f0-43ba-b077-eaa06cbd728a}
    // Type: UINT32
    // If present and nonzero, indicates that the sample is a clean point (key
    // frame), and decoding can begin at this sample.

    Makes sense, but not mentioned in documentation for any decoder, only in the header file where its defined. 

    • Edited by n808 Tuesday, April 16, 2013 12:04 AM
    Tuesday, April 16, 2013 12:03 AM