none
IMFTransform never has enough input data to process a frame

    Question

  • I am trying to play an H264 video in a Direct3D app using an IMFSourceReader and a IMFTransform (CLSID_CMSH264DecoderMFT) to do the decoding. On every D3D call to render(), I want to read a sample from the SourceReader, push the sample through the decoder, and grab the uncompressed pixels from the decoder so they can be drawn to screen. Ideally, the decoder would write the decoded frame directly into a D3D surface, but... one thing at a time. (Why am I doing this? Because I want very precise control over which frame of my video draws on every render() call.)

    So, I believe that I have properly set the input and output stream types for my decoder. My SourceReader is outputting MFVideoFormat_H264. My decoder input type is MFVideoFormat_H264. But every time I call pReader->ReadSample() and pDecoder->ProcessInput(0,pSample,0) and then pDecoder->GetOutputStatus(&flag), my flag suggests that the decoder needs more data before it can generate an output sample. This goes on for all 1800 calls to pReader->ReadSample, (by which time I've pushed about 8MB of data through my pSample -- just about the entire contents of the input video file, as expected.)

    I suspect that my first call to ReadSample() should get me a large enough sample that I can decode the first frame. After all, on the second call, the timestamp returned is no longer 0!

    ...But my decoder is never satisfied with the data I send it. It keeps wanting more. Even if I just call pDecoder->ProcessOutput(0,1,&db,&status), it returns MF_E_TRANSFORM_NEED_MORE_INPUT.

    I'm running Windows 7 64bit, but compiling as a Win32 application.

    I haven't been able to find much of anything about this decoder online except for this one msdn page:
    http://msdn.microsoft.com/en-us/library/dd797815%28VS.85%29.aspx

    The video file that I am using is an H264 wrapped in a .mov. When I explicitly tell my SourceReader to read this file and show me the samples, it properly resolves the format types and knows how to deliver the samples properly for each frame. So I know the video file is ok. I want to use my own instance of the decoder because I feel like I have better control over the MFT_OUTPUT_DATA_BUFFER and will be able to route it more easily to a texture. If this can somehow be done right in the SourceReader, I'd love to avoid messing with an IMFTransform altogether.

    Any idea why this might be happening? What details should I provide to make my issue clearer?

    Thanks so much!
    Friday, September 04, 2009 11:21 PM

Answers

  • There are a few nonintuitive things about the in box H264 decoder that might be the source of what you are seeing. 

    - The H264 decoder does not properly set the output flags, so GetOutputStatus will never return that the decoder has data. 
    - If the H264 cannot process a frame because the decoder believes the frame to be corrupt, it will silently drop the frame and request more input.  The codec guys tell me this is important for handling corrupt portions of bitstreams gracefully, but I have been confused by this myself at times.  If the media type is incorrect then the decoder might consider all frames as corrupt, but if you are just getting the source reader's native media type and setting it on the decoder then that should not happen.
    - The H264 decoder does asynchronous processing of multiple samples.  If the decoder has processing slots available and no output sample is ready yet, it will request more input.  You should try feeding an input sample and then re-calling ProcessOutput every time it returns MF_E_TRANSFORM_NEED_MORE_INPUT.  After several times it should finally give you an output frame.  If you need to get all of the data out of the decoder without providing more input -- for example, after you send the last input frame -- you can send the MFT_MESSAGE_COMMAND_DRAIN message to ProcessMessage.

    Tuesday, September 08, 2009 11:46 PM
    Moderator

All replies

  • There are a few nonintuitive things about the in box H264 decoder that might be the source of what you are seeing. 

    - The H264 decoder does not properly set the output flags, so GetOutputStatus will never return that the decoder has data. 
    - If the H264 cannot process a frame because the decoder believes the frame to be corrupt, it will silently drop the frame and request more input.  The codec guys tell me this is important for handling corrupt portions of bitstreams gracefully, but I have been confused by this myself at times.  If the media type is incorrect then the decoder might consider all frames as corrupt, but if you are just getting the source reader's native media type and setting it on the decoder then that should not happen.
    - The H264 decoder does asynchronous processing of multiple samples.  If the decoder has processing slots available and no output sample is ready yet, it will request more input.  You should try feeding an input sample and then re-calling ProcessOutput every time it returns MF_E_TRANSFORM_NEED_MORE_INPUT.  After several times it should finally give you an output frame.  If you need to get all of the data out of the decoder without providing more input -- for example, after you send the last input frame -- you can send the MFT_MESSAGE_COMMAND_DRAIN message to ProcessMessage.

    Tuesday, September 08, 2009 11:46 PM
    Moderator
  • hr = ppDecoder->GetOutputStatus(&m1);

    hr returns s_ok, and m1 returns 0. Then what does this mean?

    Monday, July 21, 2014 8:14 AM