locked
IMFSinkWriter ate my B-frames RRS feed

  • Question

  • I'm attempting to replace the audio track in an MPEG-4 file with a different one. I particularly don't want to re-encode the video, just mux a new soundtrack alongside it, and I'm using a pair of IMFSourceReaders and an IMFSinkWriter to achieve this.

    The problem I have is that while the audio part's working fine the H.264 stream I write has lost all its b-frames, which has a detrimental effect on the picture quality (!).

    My test content is 400 secs of 24fps video with closed GOPs and I frames every five seconds. I get 9601 samples from the source reader, and write 9601 samples to the sink writer, but the file I output only has 5405 entries in the stsz atom, and (according to both Elecard's StreamEye and some frame-type dumping code I wrote) they're all either I- or P-frames.

    Source material - H.264 encapsulated in MPEG-4 - looks like this:

    in our hand-rolled MPEG-4 browser and in StreamEye - note the B- frames interspersed between many of the P-frames as you'd expect. The same views of the file output by the sink writer look like this:

     

    i.e. there are far fewer samples in the target file and all the B-frames are missing.

    I'm assuming that I've managed to get something horribly wrong/miss a setting somewhere; a minimal repro is below.

    #include <windows.h>
    #include <mfapi.h>
    #include <Mfidl.h>
    #include <mfreadwrite.h>
    #include <mferror.h>
    #include <atlbase.h>
    
    #pragma comment(lib, "mfplat")
    #pragma comment(lib, "mf")
    #pragma comment(lib, "mfuuid")
    #pragma comment(lib, "mfreadwrite")
    
    #ifndef CHECK_HR
    #define CHECK_HR(val) { HRESULT hr; if (FAILED(hr = (val))) { DebugBreak(); } }
    #endif
    
    int wmain(int argc, wchar_t* argv[])
    {
        HRESULT hr = S_OK;
        CHECK_HR(CoInitialize(NULL));
        {
            CHECK_HR(MFStartup(MF_VERSION));
    
            // set up reader for the video file
            CComPtr<IMFSourceReader> videoReader;
            CHECK_HR(MFCreateSourceReaderFromURL(argv[1], NULL, &videoReader));
            CComPtr<IMFMediaType> videoMediaType;
            DWORD mediaTypeIndex = 0;
            CHECK_HR(videoReader->GetNativeMediaType(MF_SOURCE_READER_FIRST_VIDEO_STREAM, mediaTypeIndex, &videoMediaType));
            CHECK_HR(videoReader->SetCurrentMediaType(MF_SOURCE_READER_FIRST_VIDEO_STREAM, NULL, videoMediaType));
    
            // set up the sink writer for writing MPEG-4
            CComPtr<IMFAttributes> sinkWriterAttributes;
            CHECK_HR(MFCreateAttributes(&sinkWriterAttributes, 2));
            CHECK_HR(sinkWriterAttributes->SetUINT32(MF_SINK_WRITER_DISABLE_THROTTLING, 1));
            CHECK_HR(sinkWriterAttributes->SetGUID(MF_TRANSCODE_CONTAINERTYPE, MFTranscodeContainerType_MPEG4));
    
            CComPtr<IMFSinkWriter> sinkWriter;
            CHECK_HR(MFCreateSinkWriterFromURL(argv[2], NULL, sinkWriterAttributes, &sinkWriter));
    
            // add video stream
            DWORD h264StreamIndex;
            CHECK_HR(sinkWriter->AddStream(videoMediaType, &h264StreamIndex));
    
            // copy from source to target
            CHECK_HR(sinkWriter->BeginWriting());
            DWORD streamIndex = 0, flags = 0;
            LONGLONG timestamp;
            while(true)
            {
                CComPtr<IMFSample> sample;
                CHECK_HR(videoReader->ReadSample(MF_SOURCE_READER_FIRST_VIDEO_STREAM, 0, &streamIndex, &flags, &timestamp, &sample));
                if (flags & MF_SOURCE_READERF_ENDOFSTREAM)
                    break;
    
                CHECK_HR(sinkWriter->WriteSample(h264StreamIndex, sample));
            }
    
            CHECK_HR(sinkWriter->Finalize());
            (void) MFShutdown();
        }
    
        CoUninitialize();
        return FAILED(hr);
    }
    
    

    This has been built on WS2008R2; it expects the name of a MPEG-4 file containing one H.264 video stream in argv[1] and it copies the H.264 stream to the filename in argv[2]. Alternatively, mfcopy shows precisely the same behaviour.

    Does anyone have any idea what's going on or what I've missed?


    -- Jon
    Thursday, November 17, 2011 12:57 PM

Answers

All replies

  • Just wanted to give this one a bump - I have the exact same problem. A little extra information is that the MFCopy example code exhibits the same behaviour.

     

    Cheers,

     

    Tony

    Wednesday, November 23, 2011 4:14 PM
  • Probably, this explains it:

    http://msdn.microsoft.com/en-us/library/windows/desktop/dd757763(v=VS.85).aspx

    "The MPEG-4 file sink does not support content that has B frames".

    I hope I'm wrong.

    Markus

     

    • Marked as answer by JonK Thursday, November 24, 2011 2:41 PM
    Thursday, November 24, 2011 2:25 PM
  • Bugger.

    Yes, that would do it - and I completely missed that caveat. Thanks.

    Looks like I'm writing an MPEG-4 muxer.

    --

    Jon


    -- Jon
    Thursday, November 24, 2011 2:40 PM
  • Thanks, I missed that too. I was hoping to change the audio track of any supported video but this special case breaks that idea!

     

     

    Thursday, November 24, 2011 3:37 PM