locked
Video SinkWriter in Metro app. RRS feed

  • Question

  • I'm trying to write frames in a video file (Format and compression is not important yet!).
    I followed this tutorial Using the Sink Writer to Encode Video but I'm getting "Access denied" exception in MFCreateSinkWriterFromURL function while I'm sure I have the permissions in application manifest to access that file.

    I also tried to MFCreateSinkWriterFromMediaSink function and I'm getting an Invalid parameter exception (Sending NULL to pAttributes) and tried the code inUsing SinkWriter with the same error.

    I just need to put my frames in a video, no need for special compression... So are these functions really work in Metro style and my implementation is wrong or I need to do something else?



    • Edited by SiavashBD Monday, July 23, 2012 12:01 PM
    Monday, July 23, 2012 11:58 AM

Answers

  • Hi,

    I just saw your post. It's probably too late but I'll post as reference: MFCreateSinkWriterFromURL() needs to be passed both the file path (or at least the file extension) and the bytestream.

    Here is a C++ code sample creating an MP4 video with a bouncing square:

    #define _USE_MATH_DEFINES
    #include <cmath>
    #include <wrl\client.h>
    #include <ppltasks.h>
    #include <mfapi.h>
    #include <mfidl.h>
    #include <mfreadwrite.h>  
    #include "MediaWriter.h"
    
    using namespace Microsoft::WRL;
    using namespace concurrency;
    using namespace Extensions;
    using namespace Platform;
    using namespace Windows::Foundation;
    using namespace Windows::UI::Core;
    
    #define CHK(statement)	{HRESULT _hr = (statement); if (FAILED(_hr)) { throw ref new COMException(_hr); };}
    
    // Class to start and shutdown Media Foundation
    class AutoMF
    {
    public:
        AutoMF()
            : _bInitialized(false)
        {
            CHK(MFStartup(MF_VERSION));
        }
    
        ~AutoMF()
        {
            if (_bInitialized)
            {
                (void) MFShutdown();
            }
        }
    
    private:
        bool _bInitialized;
    };
    
    Windows::Foundation::IAsyncActionWithProgress<double>^ MediaWriter::WriteAsync(Windows::Storage::Streams::IRandomAccessStream^ stream)
    {
        return create_async([this, stream]
        (progress_reporter<double> reporter, cancellation_token token) {
            
            // some parameters   
            const unsigned int WIDTH = 640;   
            const unsigned int HEIGHT = 480;   
            const unsigned int RATE_NUM = 30000;   
            const unsigned int RATE_DENOM = 1000;   
            const unsigned int BITRATE = 3000000;   
            const unsigned int ASPECT_NUM = 1;   
            const unsigned int ASPECT_DENOM = 1;   
            const unsigned long  BPP_IN = 32;   
            const long long hnsSampleDuration = 10000000 * (long long)RATE_DENOM / (long long)RATE_NUM;   
            const unsigned long cbMaxLength = WIDTH * HEIGHT * BPP_IN / 8; 
            const unsigned int ONE_SECOND = RATE_NUM / RATE_DENOM;
            const unsigned int FRAME_NUM = 10 * ONE_SECOND;
    
            AutoMF mf;
    
            //
            // Create the Sink Writer
            //
            
            ComPtr<IMFByteStream> spByteStream;
            CHK(MFCreateMFByteStreamOnStreamEx((IUnknown*)stream, &spByteStream));
    
            ComPtr<IMFAttributes> spAttr;
            CHK(MFCreateAttributes(&spAttr, 10));
            CHK(spAttr->SetUINT32(MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS, true));
    
            ComPtr<IMFSinkWriter> spSinkWriter;
            CHK(MFCreateSinkWriterFromURL(L".mp4", spByteStream.Get(), spAttr.Get(), &spSinkWriter));
    
            //   
            // Setup the output media type   
            //   
    
            ComPtr<IMFMediaType> spTypeOut;  
            CHK(MFCreateMediaType(&spTypeOut));   
            CHK(spTypeOut->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video));     
            CHK(spTypeOut->SetGUID(MF_MT_SUBTYPE, MFVideoFormat_H264));   
            CHK(spTypeOut->SetUINT32(MF_MT_AVG_BITRATE, BITRATE));   
            CHK(spTypeOut->SetUINT32(MF_MT_INTERLACE_MODE, MFVideoInterlace_Progressive));   
            CHK(MFSetAttributeSize(spTypeOut.Get(), MF_MT_FRAME_SIZE, WIDTH, HEIGHT));   
            CHK(MFSetAttributeRatio(spTypeOut.Get(), MF_MT_FRAME_RATE, RATE_NUM, RATE_DENOM));
            CHK(MFSetAttributeRatio(spTypeOut.Get(), MF_MT_PIXEL_ASPECT_RATIO, ASPECT_NUM, ASPECT_DENOM));
    
            DWORD streamIndex;     
            CHK(spSinkWriter->AddStream(spTypeOut.Get(), &streamIndex));   
         
            //   
            // Setup the input media type   
            //   
    
            ComPtr<IMFMediaType> spTypeIn;
            CHK(MFCreateMediaType(&spTypeIn));   
            CHK(spTypeIn->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video));   
            CHK(spTypeIn->SetGUID(MF_MT_SUBTYPE, MFVideoFormat_RGB32));     
            CHK(spTypeIn->SetUINT32(MF_MT_INTERLACE_MODE, MFVideoInterlace_Progressive));   
            CHK(MFSetAttributeSize(spTypeIn.Get(), MF_MT_FRAME_SIZE, WIDTH, HEIGHT) );   
            CHK(MFSetAttributeRatio(spTypeIn.Get(), MF_MT_FRAME_RATE, RATE_NUM, RATE_DENOM));   
            CHK(MFSetAttributeRatio(spTypeIn.Get(), MF_MT_PIXEL_ASPECT_RATIO, ASPECT_NUM, ASPECT_DENOM));   
    
            CHK(spSinkWriter->SetInputMediaType(streamIndex, spTypeIn.Get(), nullptr));   
         
            //   
            // Write some data   
            //   
    
            CHK(spSinkWriter->BeginWriting());   
    
            double progress = 0.;
            LONGLONG hnsSampleTime = 0;   
            for (unsigned int nFrame = 0; nFrame < FRAME_NUM; nFrame++)   
            {   
                if (token.is_canceled())
                {
                    break;
                }
    
                double newProgress = 100. * (double)nFrame / (double)FRAME_NUM;
                if (newProgress - progress >= 1.)
                {
                    progress = newProgress;
                    reporter.report(progress);
                }
    
                //   
                // Create a media sample   
                //   
    
                ComPtr<IMFSample> spSample;   
                CHK(MFCreateSample(&spSample));   
                CHK(spSample->SetSampleDuration(hnsSampleDuration));   
                CHK(spSample->SetSampleTime(hnsSampleTime));   
                hnsSampleTime += hnsSampleDuration;   
    
                //   
                // Add a media buffer filled with random data   
                //   
    
                ComPtr<IMFMediaBuffer> spBuffer;   
                CHK(MFCreateMemoryBuffer(cbMaxLength, &spBuffer));   
                CHK(spBuffer->SetCurrentLength(cbMaxLength));   
                CHK(spSample->AddBuffer(spBuffer.Get()));   
    
                // Draw a bouncing white rectangle over black background
                unsigned char *pbBuffer = nullptr;     
                CHK(spBuffer->Lock(&pbBuffer, nullptr, nullptr));   
                for (unsigned int i=0; i < HEIGHT; i++)
                {
                    for (unsigned int j=0; j < WIDTH; j++)   
                    {   
                        unsigned int pos = 4 * (i * WIDTH + j);
                        unsigned char val = 255 * (
                            (abs((int)WIDTH / 2 - (int)j) < (WIDTH / 4)) && 
                            (abs(HEIGHT * (.5 + .1 * sin(2. * M_PI * (double)nFrame / (double)ONE_SECOND)) - (int)i) < (HEIGHT / 4))
                            );
                        pbBuffer[pos  ] = val;
                        pbBuffer[pos+1] = val;
                        pbBuffer[pos+2] = val;
                        pbBuffer[pos+3] = val;
                    }
                }
                CHK(spBuffer->Unlock());   
    
                //   
                // Write the media sample   
                //   
    
                CHK(spSinkWriter->WriteSample(streamIndex, spSample.Get()));   
            }   
    
            if (!token.is_canceled())
            {
                CHK(spSinkWriter->Finalize());   
    
                reporter.report(100.);
            }
        });
    }

    and a JavaScript code sample calling it:

        var stream = null;
        Windows.Storage.KnownFolders.videosLibrary.createFileAsync("out.mp4", Windows.Storage.CreationCollisionOption.generateUniqueName).then(function (file) {
            if (file) {
    
                file.openAsync(Windows.Storage.FileAccessMode.readWrite).then(function (newStream) {
    
                    stream = newStream;
    
                    var writer = new Extensions.MediaWriter();
                    return writer.writeAsync(stream);
    
                }).then(function () {
    
                    stream.close();
                    stream = null;
    
                    var player = document.getElementById('player');
                    player.src = URL.createObjectURL(file, { oneTimeOnly: true });
                    player.play();
    
                }, function (error) {
    
                    stream.close();
                    stream = null;
    
                }, function (progress) {
    
                    document.getElementById('progressbar').value = progress;
    
                });
            }
        });
    

     
    • Marked as answer by SiavashBD Friday, September 21, 2012 10:21 AM
    Tuesday, September 11, 2012 1:00 AM

All replies

  • Hello SiavashBD,

    The Metro environment does not really have the concept of a URL for local files. You will need to open a WinRT file stream. You will then need pass NULL for the URL and pass a valid byte stream to the MFCreateSinkWriterFrom URL. Once the data has been written to your byte stream you can copy this data into the WinRT file stream and flush it to disk. 

    I hope this helps,

    James 


    Windows Media SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

    Wednesday, July 25, 2012 1:05 AM
    Moderator
  • Hi James,

    Thanks for your response. I also tried like this:

    hr = ( CoInitializeEx(0, COINIT_MULTITHREADED)); hr = ( MFStartup(MF_VERSION) ); //MFCreateSinkWriterFromURL + URL //hr = MFCreateSinkWriterFromURL(L"C:\\Users\\Siavash\\Pictures\\output.wmv", NULL, NULL, &pWriter); create_task(KnownFolders::PicturesLibrary->CreateFileAsync("output.wmv",CreationCollisionOption::GenerateUniqueName)).then([this](StorageFile^ file) { create_task(file->OpenAsync(FileAccessMode::ReadWrite)).then([this](Streams::IRandomAccessStream^ storageStream) { HRESULT hr = NULL; IMFReadWriteClassFactory *pClassFactory = NULL; IMFSinkWriter *pWriter = NULL; IMFMediaType *pVideoOutType = NULL; IMFByteStream *pByteStream = NULL; IMFAttributes *pAttributes = NULL; IMFMediaSink * pMediaSink = NULL; hr = MFCreateMFByteStreamOnStreamEx((IUnknown*)storageStream, &pByteStream); if(hr != S_OK) return 1; //MFCreateSinkWriterFromURL + IMFByteStream hr = MFCreateAttributes(&pAttributes, 1); //hr = pAttributes->SetUINT32( MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS, false ); //hr = pAttributes->SetUINT32( MF_READWRITE_DISABLE_CONVERTERS , false ); hr = MFCreateSinkWriterFromURL(NULL, pByteStream,pAttributes,&pWriter); // Invalid argument!!! /* //IMFReadWriteClassFactory ( hr = CoCreateInstance( CLSID_MFReadWriteClassFactory, NULL, CLSCTX_INPROC_SERVER, IID_PPV_ARGS( &pClassFactory ) ), L"Error creating read write class factory." ); (hr = pClassFactory->CreateInstanceFromObject( CLSID_MFSinkWriter,pByteStream , pVideoOutType, IID_PPV_ARGS( &pWriter )), L"Error creating sink writer from custom byte stream." ); */

    				});

    }); });

    And I'm getting "E_INVALIDARG One or more arguments are invalid." as HRESULT.. I'm sure it's because of the attributes but I have no idea how else should it be.

    The MFCreateSinkWriterFromURL API needs MF_TRANSCODE_CONTAINERTYPEto determine the type of file but it seems that it's undefined in Metro style so I couldn't set it.

    I also tried getting the attributes from another source:

     hr = pByteStream->QueryInterface(__uuidof(IMFAttributes),(void**)&pAttrib);

    but again the same error (E_INVALIDARG)

    • Edited by SiavashBD Wednesday, July 25, 2012 11:59 AM
    Wednesday, July 25, 2012 7:28 AM
  • Hi,

    I just saw your post. It's probably too late but I'll post as reference: MFCreateSinkWriterFromURL() needs to be passed both the file path (or at least the file extension) and the bytestream.

    Here is a C++ code sample creating an MP4 video with a bouncing square:

    #define _USE_MATH_DEFINES
    #include <cmath>
    #include <wrl\client.h>
    #include <ppltasks.h>
    #include <mfapi.h>
    #include <mfidl.h>
    #include <mfreadwrite.h>  
    #include "MediaWriter.h"
    
    using namespace Microsoft::WRL;
    using namespace concurrency;
    using namespace Extensions;
    using namespace Platform;
    using namespace Windows::Foundation;
    using namespace Windows::UI::Core;
    
    #define CHK(statement)	{HRESULT _hr = (statement); if (FAILED(_hr)) { throw ref new COMException(_hr); };}
    
    // Class to start and shutdown Media Foundation
    class AutoMF
    {
    public:
        AutoMF()
            : _bInitialized(false)
        {
            CHK(MFStartup(MF_VERSION));
        }
    
        ~AutoMF()
        {
            if (_bInitialized)
            {
                (void) MFShutdown();
            }
        }
    
    private:
        bool _bInitialized;
    };
    
    Windows::Foundation::IAsyncActionWithProgress<double>^ MediaWriter::WriteAsync(Windows::Storage::Streams::IRandomAccessStream^ stream)
    {
        return create_async([this, stream]
        (progress_reporter<double> reporter, cancellation_token token) {
            
            // some parameters   
            const unsigned int WIDTH = 640;   
            const unsigned int HEIGHT = 480;   
            const unsigned int RATE_NUM = 30000;   
            const unsigned int RATE_DENOM = 1000;   
            const unsigned int BITRATE = 3000000;   
            const unsigned int ASPECT_NUM = 1;   
            const unsigned int ASPECT_DENOM = 1;   
            const unsigned long  BPP_IN = 32;   
            const long long hnsSampleDuration = 10000000 * (long long)RATE_DENOM / (long long)RATE_NUM;   
            const unsigned long cbMaxLength = WIDTH * HEIGHT * BPP_IN / 8; 
            const unsigned int ONE_SECOND = RATE_NUM / RATE_DENOM;
            const unsigned int FRAME_NUM = 10 * ONE_SECOND;
    
            AutoMF mf;
    
            //
            // Create the Sink Writer
            //
            
            ComPtr<IMFByteStream> spByteStream;
            CHK(MFCreateMFByteStreamOnStreamEx((IUnknown*)stream, &spByteStream));
    
            ComPtr<IMFAttributes> spAttr;
            CHK(MFCreateAttributes(&spAttr, 10));
            CHK(spAttr->SetUINT32(MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS, true));
    
            ComPtr<IMFSinkWriter> spSinkWriter;
            CHK(MFCreateSinkWriterFromURL(L".mp4", spByteStream.Get(), spAttr.Get(), &spSinkWriter));
    
            //   
            // Setup the output media type   
            //   
    
            ComPtr<IMFMediaType> spTypeOut;  
            CHK(MFCreateMediaType(&spTypeOut));   
            CHK(spTypeOut->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video));     
            CHK(spTypeOut->SetGUID(MF_MT_SUBTYPE, MFVideoFormat_H264));   
            CHK(spTypeOut->SetUINT32(MF_MT_AVG_BITRATE, BITRATE));   
            CHK(spTypeOut->SetUINT32(MF_MT_INTERLACE_MODE, MFVideoInterlace_Progressive));   
            CHK(MFSetAttributeSize(spTypeOut.Get(), MF_MT_FRAME_SIZE, WIDTH, HEIGHT));   
            CHK(MFSetAttributeRatio(spTypeOut.Get(), MF_MT_FRAME_RATE, RATE_NUM, RATE_DENOM));
            CHK(MFSetAttributeRatio(spTypeOut.Get(), MF_MT_PIXEL_ASPECT_RATIO, ASPECT_NUM, ASPECT_DENOM));
    
            DWORD streamIndex;     
            CHK(spSinkWriter->AddStream(spTypeOut.Get(), &streamIndex));   
         
            //   
            // Setup the input media type   
            //   
    
            ComPtr<IMFMediaType> spTypeIn;
            CHK(MFCreateMediaType(&spTypeIn));   
            CHK(spTypeIn->SetGUID(MF_MT_MAJOR_TYPE, MFMediaType_Video));   
            CHK(spTypeIn->SetGUID(MF_MT_SUBTYPE, MFVideoFormat_RGB32));     
            CHK(spTypeIn->SetUINT32(MF_MT_INTERLACE_MODE, MFVideoInterlace_Progressive));   
            CHK(MFSetAttributeSize(spTypeIn.Get(), MF_MT_FRAME_SIZE, WIDTH, HEIGHT) );   
            CHK(MFSetAttributeRatio(spTypeIn.Get(), MF_MT_FRAME_RATE, RATE_NUM, RATE_DENOM));   
            CHK(MFSetAttributeRatio(spTypeIn.Get(), MF_MT_PIXEL_ASPECT_RATIO, ASPECT_NUM, ASPECT_DENOM));   
    
            CHK(spSinkWriter->SetInputMediaType(streamIndex, spTypeIn.Get(), nullptr));   
         
            //   
            // Write some data   
            //   
    
            CHK(spSinkWriter->BeginWriting());   
    
            double progress = 0.;
            LONGLONG hnsSampleTime = 0;   
            for (unsigned int nFrame = 0; nFrame < FRAME_NUM; nFrame++)   
            {   
                if (token.is_canceled())
                {
                    break;
                }
    
                double newProgress = 100. * (double)nFrame / (double)FRAME_NUM;
                if (newProgress - progress >= 1.)
                {
                    progress = newProgress;
                    reporter.report(progress);
                }
    
                //   
                // Create a media sample   
                //   
    
                ComPtr<IMFSample> spSample;   
                CHK(MFCreateSample(&spSample));   
                CHK(spSample->SetSampleDuration(hnsSampleDuration));   
                CHK(spSample->SetSampleTime(hnsSampleTime));   
                hnsSampleTime += hnsSampleDuration;   
    
                //   
                // Add a media buffer filled with random data   
                //   
    
                ComPtr<IMFMediaBuffer> spBuffer;   
                CHK(MFCreateMemoryBuffer(cbMaxLength, &spBuffer));   
                CHK(spBuffer->SetCurrentLength(cbMaxLength));   
                CHK(spSample->AddBuffer(spBuffer.Get()));   
    
                // Draw a bouncing white rectangle over black background
                unsigned char *pbBuffer = nullptr;     
                CHK(spBuffer->Lock(&pbBuffer, nullptr, nullptr));   
                for (unsigned int i=0; i < HEIGHT; i++)
                {
                    for (unsigned int j=0; j < WIDTH; j++)   
                    {   
                        unsigned int pos = 4 * (i * WIDTH + j);
                        unsigned char val = 255 * (
                            (abs((int)WIDTH / 2 - (int)j) < (WIDTH / 4)) && 
                            (abs(HEIGHT * (.5 + .1 * sin(2. * M_PI * (double)nFrame / (double)ONE_SECOND)) - (int)i) < (HEIGHT / 4))
                            );
                        pbBuffer[pos  ] = val;
                        pbBuffer[pos+1] = val;
                        pbBuffer[pos+2] = val;
                        pbBuffer[pos+3] = val;
                    }
                }
                CHK(spBuffer->Unlock());   
    
                //   
                // Write the media sample   
                //   
    
                CHK(spSinkWriter->WriteSample(streamIndex, spSample.Get()));   
            }   
    
            if (!token.is_canceled())
            {
                CHK(spSinkWriter->Finalize());   
    
                reporter.report(100.);
            }
        });
    }

    and a JavaScript code sample calling it:

        var stream = null;
        Windows.Storage.KnownFolders.videosLibrary.createFileAsync("out.mp4", Windows.Storage.CreationCollisionOption.generateUniqueName).then(function (file) {
            if (file) {
    
                file.openAsync(Windows.Storage.FileAccessMode.readWrite).then(function (newStream) {
    
                    stream = newStream;
    
                    var writer = new Extensions.MediaWriter();
                    return writer.writeAsync(stream);
    
                }).then(function () {
    
                    stream.close();
                    stream = null;
    
                    var player = document.getElementById('player');
                    player.src = URL.createObjectURL(file, { oneTimeOnly: true });
                    player.play();
    
                }, function (error) {
    
                    stream.close();
                    stream = null;
    
                }, function (progress) {
    
                    document.getElementById('progressbar').value = progress;
    
                });
            }
        });
    

     
    • Marked as answer by SiavashBD Friday, September 21, 2012 10:21 AM
    Tuesday, September 11, 2012 1:00 AM
  • Hi Is it possible to get a working sample of this?
    Saturday, November 10, 2012 10:01 AM
  • Hi

    I am getting this error message any idea what this is?

    Error 17 error LNK2001: unresolved external symbol _MF_MT_FRAME_RATE

    Thanks

    Saturday, November 10, 2012 12:25 PM
  • You need to add the appropriate .lib files as additional link options in the RT component project properties. IIRC, the 3 you'll need for a sample like this are:

    mfplat.lib

    mfreadwrite.lib

    mfuuid.lib

    The additional link options are config specific, so you'll probably need to add them to both the Release and Debug targets.

    Thursday, January 3, 2013 7:36 PM
  • The video created by the code given here by Matthieu doesn't play in all media players. It doesn't play in QuickTime and VLC on Win8, and not in Windows Media Player on Win7. When uploading to YouTube and then downloading again results in a mp4 file, which playes everywhere.

    So there is something missing in the sample, but what?

    Ronald

    Sunday, March 17, 2013 10:13 AM
  • Yes, I also met this problem. Is it a bug for IMFSinkWriter on winRT? 
    Monday, May 13, 2013 2:56 AM
  • Hi: I use the code to test and find if write thousands of samples to video(1280X720), it spend too many times. for 1min video it will spend about 5min.  but for a diretshow implementation on desktop or for capture video from camera it will fast.  So do you have any advise on this result?
    Thursday, May 16, 2013 7:54 AM
  • I created a new video from one or more existing but not how to add a audio this.
    Wednesday, July 9, 2014 5:08 PM