locked
Hardware accelerated SinkWriter Example RRS feed

  • Question

  • Does anyone know of, or have a sample of their own they will share, that illustrates how to use MF to create and use a SinkWriter to hardware encode NV12 input to h264?

    I can get my code working in software, but I need to utilize hardware encoding resources to lower my cpu usage.  I am working on a Dell Latitude E6450 with both an AMD Radeon HD 8790M, and an Intel Core I7-4800MQ CPU. Both are supposed to support hardware encoding, and have MFTs. 

    I would greatly appreciate tips/advice/code samples to correct this problem.

     mftrace shows the following:

    For the amd gpu:

    11308,EB0 15:22:09.34586 COle32ExportDetours::CoCreateInstance @ Failed to create {ADC9BC80-0F41-46C6-AB75-D693D793597D} AMD H.264 Hardware MFT Encoder (C:\Program Files\Common Files\ATI Technologies\Multimedia\AMDh264Enc32.dll) hr=0x80004005 E_FAIL
    11308,EB0 15:22:09.34587 COle32ExportDetours::CoCreateInstance @ call to 'hr' failed (hr=0x80004005) at avcore\mf\samples\mf360\mftrace\mfdetours\otherdetours\ole32exportdetours.cpp:116
    11308,EB0 15:22:09.34587 COle32ExportDetours::CoCreateInstance @ - exit (failed hr=0x80004005 E_FAIL)
    11308,EB0 15:22:09.34587 CMFActivateDetours::ActivateObject @0078D1C0 call to 'FindDetouredVtbl( This->lpVtbl )->ActivateObject( This, riid, ppv )' failed (hr=0x80004005) at avcore\mf\samples\mf360\mftrace\mfdetours\interfacedetours\mfactivatedetours.cpp:440
    11308,EB0 15:22:09.34587 CMFActivateDetours::ActivateObject @0078D1C0 - exit (failed hr=0x80004005 E_FAIL)

    For the Intel cpu/gpu:

    6332,CA4 15:24:04.14466 COle32ExportDetours::CoCreateInstance @ Created {4BE8D3C0-0515-4A37-AD55-E4BAE19AF471} Intel® Quick Sync Video H.264 Encoder MFT (C:\Program Files\Intel\Media SDK\mfx_mft_h264ve_w7_32.dll) @07AD3A00 - traced interfaces: IMFTransform @07AD3A00,
    6332,CA4 15:24:04.14466 COle32ExportDetours::CoCreateInstance @ - exit
    6332,CA4 15:24:04.14466 CMFActivateDetours::GetGUID @0040D218 - enter
    6332,CA4 15:24:04.14466 CMFActivateDetours::GetGUID @0040D218 - exit
    6332,CA4 15:24:04.14466 CMFActivateDetours::GetUnknown @0040D218 - enter
    6332,CA4 15:24:04.14466 CMFActivateDetours::GetUnknown @0040D218 attribute not found guidKey = MFT_FIELDOFUSE_UNLOCK_Attribute
    6332,CA4 15:24:04.14611 CMFActivateDetours::GetUnknown @0040D218 - exit (failed hr=0xC00D36E6 MF_E_ATTRIBUTENOTFOUND)
    6332,CA4 15:24:04.14611 CMFActivateDetours::GetGUID @0040D218 - enter
    6332,CA4 15:24:04.14611 CMFActivateDetours::GetGUID @0040D218 - exit
    6332,CA4 15:24:04.14611 CMFActivateDetours::GetUnknown @0040D218 - enter
    6332,CA4 15:24:04.14611 CMFActivateDetours::GetUnknown @0040D218 attribute not found guidKey = MFT_PREFERRED_ENCODER_PROFILE
    6332,CA4 15:24:04.14611 CMFActivateDetours::GetUnknown @0040D218 - exit (failed hr=0xC00D36E6 MF_E_ATTRIBUTENOTFOUND)
    6332,CA4 15:24:04.14611 CMFActivateDetours::GetUnknown @0040D218 - enter
    6332,CA4 15:24:04.14612 CMFActivateDetours::GetUnknown @0040D218 attribute not found guidKey = MFT_PREFERRED_OUTPUTTYPE_Attribute
    6332,CA4 15:24:04.14612 CMFActivateDetours::GetUnknown @0040D218 - exit (failed hr=0xC00D36E6 MF_E_ATTRIBUTENOTFOUND)
    6332,CA4 15:24:04.14612 CMFActivateDetours::GetUINT32 @0040D218 - enter
    6332,CA4 15:24:04.14612 CMFActivateDetours::GetUINT32 @0040D218 - exit
    6332,CA4 15:24:04.14612 CMFPlatExportDetours::MFGetMFTMerit @ - enter
    6332,CA4 15:24:04.19958 CMFPlatExportDetours::MFGetMFTMerit @ Merit validation failed for MFT @07AD3A00 (hr=80004005 E_FAIL)
    6332,CA4 15:24:04.19958 CMFPlatExportDetours::MFGetMFTMerit @ - exit (failed hr=0x80004005 E_FAIL)
    6332,CA4 15:24:04.20529 CMFActivateDetours::ActivateObject @0040D218 call to 'FindDetouredVtbl( This->lpVtbl )->ActivateObject( This, riid, ppv )' failed (hr=0x80004005) at avcore\mf\samples\mf360\mftrace\mfdetours\interfacedetours\mfactivatedetours.cpp:440
    6332,CA4 15:24:04.20529 CMFActivateDetours::ActivateObject @0040D218 - exit (failed hr=0x80004005 E_FAIL)

    Friday, June 5, 2015 3:26 PM

All replies

  • Hi,

    I have managed to get both AMD and Intel Quick Sync to work, but be aware:

    - You can't use the MFTrace tool because then this will always happen: MFGetMFTMerit @ Merit validation failed for MFT @07AD3A00 (hr=80004005 E_FAIL). Tip: Debug print by yourself instead.

    - I have only managed to get the Quick sync MFT to work with Win8.1, not Win7 (does not process samples)

    Try doing something with the IClassFactory, described in this thread:

    https://social.msdn.microsoft.com/Forums/windowshardware/en-US/6da521e9-7bb3-4b79-a2b6-b31509224638/win7-h264-encoder-imfsinkwriter-cant-use-quality-vbr-encoding?forum=mediafoundationdevelopment

     To get more control over the MFT creation.

    // Carl

    Monday, June 8, 2015 2:54 PM
  • Hi,

    do you want

    A) System Memory -> Hardware Encoder -> System Memory

    or

    B) Hardware Memory - >Hardware Encoder -> System Memory

    ->

    Important questions :

    1. Where does your NV12 source comes from ? System Memory ? Hardware Memory ?

    2. If it comes from hardware memory, is it from the same device the encoder is from ?

    <-

    A) 

    Use the MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS flag with set to TRUE. To encode a surface, simply pass your samples created with MFCreateDXSurfaceBuffer or MFCreateDXGISurfaceBuffer to the WriteSample method on the Sink Writer.

    B)

    Use the MF_READWRITE_ENABLE_HARDWARE_TRANSFORMS flag with set to TRUE and pass your D3D Manager with the MF_SINK_WRITER_D3D_MANAGER attribute. To encode surfaces from default pool use a shared render target and a D3D Manager on which you lock your Video Device when creating a sample ( copy of the shared render target ).

    ->

    NOTES :

    1.

    If you want to configure a H.264 or HEVC hardware encoder and your program must be able to run on Windows 7 you will encounter a problem as MF_SINK_WRITER_ENCODER_CONFIG only works starting with Windows 8. If you want to configure a hardware encoder in Windows 7 you have to implement a Class Factory, persskog already posted the link. I also posted there on how to use the Sink Writer Encoder Config because the MSDN doc is wrong.

    2.

    The performance and possible advantage with hardware encoding depends on the scenario ( A or B ). If your NV12 source is from System Memory then you wouldnt gain much with a Hardware Encoder. It would only make sense on very large files, but with small files or live sources you wouldnt get any speed advantage. If you have a D3D live source ( IMFMediaSourceEx ) which delivers from the same device ( hardware memory ) where the encoder is on then you will get a huge advantage. But if the live source delivers from a different device then you would even decrease performance as the encoder has to wait for frames to become available and that makes the System Memory to Hardware and then back again even worse.

    3.

    For D3D 10/11/12 the IDXGIDeviceManager will work flawlessly starting with Windows 8, but if you plan to do this kind of encoding in D3D9 with the IDirect3DDeviceManager9 on Windows 7 you might get disappointed with some encoders as not all vendors support D3D9 under Win 7. Check MF_SA_D3D_AWARE for D3D 9 and MF_SA_D3D11_AWARE for D3D 11 on the Hardware MFT.

    <-

    This plus the link persskog posted should help you to solve your problem.

    Regards,

    Francis





















    • Proposed as answer by Francis Grave Wednesday, July 1, 2015 12:47 PM
    • Unproposed as answer by Francis Grave Saturday, June 4, 2016 1:49 PM
    • Proposed as answer by Francis Grave Saturday, June 4, 2016 5:58 PM
    • Edited by Francis Grave Saturday, June 4, 2016 6:22 PM Edit
    Wednesday, July 1, 2015 12:42 PM
  • This is very helpful information.  

    I am seeing this same behavior described with the Intel Quick Sync MFT on Win7,  the error I eventually get while pushing samples to the SinkWriter is is MF_E_UNEXPECTED( 0xC00D36BB )

    If anyone figures out any workarounds for Win7, please post.
    Friday, June 3, 2016 6:04 PM
  • I had to edit my last post as there was some misleading information in it. Here is an earlier post from me in which i answered how software and hardware encoding of D3D surfaces work on a Sink Writer :

    https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/172e0d65-756a-4ab9-aef6-7cd80d15a66c/behavior-of-the-sink-writer-and-microsofts-h-264-encoder-when-it-comes-to-encoding-d3d-surfaces?forum=mediafoundationdevelopment

    If you plan to encode D3D surfaces from default pool i strongly advise you to invest into Media Session and component wrapping as the Sink Writer is very limited when it comes mixing and editing. Somewhat 3-4 years ago i was testing around with the Sink Writer but rapidly changed to using Media Sessions as i encountered to much barriers with the Sink Writer class.

    The decision if the Sink Writer is enough for your task depends on what your scenario is and what memory you are encoding from ( see last post ).

    Regards,

    Francis



    Saturday, June 4, 2016 4:52 PM
  • I know this is old but I'm hoping someone can help. I'm using Windows 10 and attempting to encode the output of the Desktop Duplication API. I tried to just pass in the output from MFCreateDXGISurfaceBuffer but the sink writer finalize method just comes with the following error message 

    hr 0xc00d4a44 : The operation failed because no samples were processed by the sink. HRESULT

    Can anyone help I've been pounding away at this for hours with no luck. When I look at the buffer length it's set to 0. Not sure if I need to do anything special or create a special ID3D11Texture2D to be passed to the sink writer. I'm attempting to do this with hardware encoding and I'm using the demos provided by MSDN as the source for everything I'm trying to do. 

    CComPtr<IMFMediaBuffer> pMediaBuffer;
    			hr = MFCreateDXGISurfaceBuffer(__uuidof(ID3D11Texture2D),Data.Frame,0,FALSE,&pMediaBuffer);
    
    			CComPtr<IMFSample> pIMFSample;
    			hr = MFCreateSample(&pIMFSample);
    			hr = pIMFSample->AddBuffer(pMediaBuffer);
    
    			ScreenVideoEncoder.WriteFrame(pSinkWriter, stream, rtStart, pIMFSample);
    

    Then my write frame method does the following: 

    HRESULT CScreenVideoEncoder::WriteFrame(
    	IMFSinkWriter *pWriter,
    	DWORD streamIndex,
    	const LONGLONG& rtStart,      // Time stamp.
    	IMFSample *pSample
    	)
    {
    	//IMFMediaBuffer *pBuffer = NULL;
    
    	const LONG cbWidth = 4 * VIDEO_WIDTH;
    	const DWORD cbBuffer = cbWidth * VIDEO_HEIGHT;
    
    	BYTE *pData = NULL;
    
    	//// Create a new memory buffer.
    	//HRESULT hr = MFCreateMemoryBuffer(cbBuffer, &pBuffer);
    	//IMFMediaBuffer *pBuffer = nullptr;
    	//HRESULT hr = pSample->GetBufferByIndex(0, &pBuffer);
    	//// Lock the buffer and copy the video frame to the buffer.
    	//if (SUCCEEDED(hr))
    	//{
    	//	hr = pBuffer->Lock(&pData, NULL, NULL);
    	//}
    	//if (SUCCEEDED(hr))
    	//{
    	//	hr = MFCopyImage(
    	//		pData,                      // Destination buffer.
    	//		cbWidth,                    // Destination stride.
    	//		(BYTE*)videoFrameBuffer,    // First row in source image.
    	//		cbWidth,                    // Source stride.
    	//		cbWidth,                    // Image width in bytes.
    	//		VIDEO_HEIGHT                // Image height in pixels.
    	//	);
    	//}
    	//if (pBuffer)
    	//{
    	//	pBuffer->Unlock();
    	//}
    
    	//// Set the data length of the buffer.
    	//if (SUCCEEDED(hr))
    	//{
    	//	hr = pBuffer->SetCurrentLength(cbBuffer);
    	//}
    
    	//// Create a media sample and add the buffer to the sample.
    	//if (SUCCEEDED(hr))
    	//{
    	//	hr = MFCreateSample(&pSample);
    	//}
    	//if (SUCCEEDED(hr))
    	//{
    	//	hr = pSample->AddBuffer(pBuffer);
    	//}
    
    	// Set the time stamp and the duration.
    	//if (SUCCEEDED(hr))
    	//{
    	 HRESULT hr = pSample->SetSampleTime(rtStart);
    	//}
    	if (SUCCEEDED(hr))
    	{
    		hr = pSample->SetSampleDuration(VIDEO_FRAME_DURATION);
    	}
    	// Send the sample to the Sink Writer.
    	if (SUCCEEDED(hr))
    	{
    		hr = pWriter->WriteSample(streamIndex, pSample);
    	}
    
    	//SafeRelease(&pSample);
    	//SafeRelease(&pBuffer);
    	return hr;
    }

    Tuesday, September 12, 2017 12:44 AM
  • Hi,

    I think you should create a IMFSample backed by a ID3D11Texture like this:

     ComPtr<IMFMediaBuffer> dxgiSurfaceBuffer;
    
    long hr = ::MFCreateDXGISurfaceBuffer(
                __uuidof(ID3D11Texture2D),
                texture2D,
                0,
                FALSE,
                set( dxgiSurfaceBuffer ) );
    
    if (SUCCEEDED(hr))
    {
        /* D3D11 video samples, the surface argument should be
           nullptr, because the DXGI surface buffer is added afterwards. */
        hr = ::MFCreateVideoSampleFromSurface( nullptr, outSample);
    }
    
    if (SUCCEEDED(hr))
    {
        hr = (*outSample)->AddBuffer( get(dxgiSurfaceBuffer) );
    }
    

    When you need to read from it:

    ComPtr<IMFMediaBuffer> mediaBuffer;
    
    ...
    
    //  QueryInterface
    auto buffer2D = mediaBuffer.As<IMF2DBuffer2>();
    
    
    frameBuffer2D->Lock2DSize(
        MF2DBuffer_LockFlags_Read,
        &dstScan0,
        &dstPitch,
        &dstStart,
        &dstLength
    );
    
    ::MFCopyImage( dstScan0, dstPitch, srcBuffer, dstPitch, dstPitch, lines);
    
    frameBuffer2D->Unlock2D();
    // Carl


    Tuesday, September 12, 2017 7:15 AM