locked
MFCreateDXGISurfaceBuffer() fails RRS feed

  • Question

  • Hi,

    i tryed to encode a ID3D11Texture2D by using MFCreateDXGISurfaceBuffer and then add the recieved buffer to an IMFSample and send it to the sink writer. My Problem is that the final video, although it has sound and records normal, has only black image.

    On the MSDN documentary of MFCreateDXGISurfaceBuffer under community additions Stewart Tootill sounds like it would work as good as MFCreateDXSurfaceBuffer does when encoding a d3d surfaces with the sink writer. The following sample code is simple and very straightforward :

    	// Create the DirectX surface buffer
    	if(SUCCEEDED(hr))hr = MFCreateDXGISurfaceBuffer(__uuidof(ID3D11Texture2D), pFrameBuffer, 0, false, &pBuffer);
    	if(SUCCEEDED(hr))hr = pBuffer->QueryInterface(__uuidof(IMF2DBuffer), (void**)&p2DBuffer);
    	if(SUCCEEDED(hr))hr = p2DBuffer->GetContiguousLength(&Length);
    
        // Set the data length of the buffer
        if(SUCCEEDED(hr))hr = pBuffer->SetCurrentLength(Length);
    
        // Create a media sample and add the buffer to the sample
        if(SUCCEEDED(hr))hr = MFCreateSample(&pSample);
        if (SUCCEEDED(hr))hr = pSample->AddBuffer(pBuffer);
    	
        // Set the time stamp and the duration
        if(SUCCEEDED(hr))hr = pSample->SetSampleTime(frameStart);
        if(SUCCEEDED(hr))hr = pSample->SetSampleDuration(frameDuration);
    
        // Send the sample to the Sink Writer
        if(SUCCEEDED(hr))hr = pSinkWriter->WriteSample(videoStream, pSample);

    My guess is that it has to do with the creation parameters of the ID3D11Texture2D. Its usage is staging and has cpu read and write access, and when i print pFrameBuffer with another API to a jpg or bmp file directly before

    if(SUCCEEDED(hr))hr = MFCreateDXGISurfaceBuffer(__uuidof(ID3D11Texture2D), pFrameBuffer, 0, false, &pBuffer);

    the texture is normal. The Format of pFrameBuffer is DXGI_FORMAT_R8G8B8A8_UNORM

    Any ideas ?




    Tuesday, November 12, 2013 8:32 AM

Answers

  • Well after testing a BGRA Format now the function succeeds...

    Now heres my problem :

    I am not the owner of the source texture. I am hooking games and extracting their backbuffers and so i have no control over the creation parameters of the textures.

    One way i see is to build up a 2nd render target and set it via OMSetRenderTargets(). Would it slow down the fps significantly when i render to 2 different targets ? Well i will see that tomorrow when i start testing this out.

    If someone at Microsoft reads this, please we need format conversions in either MFCopyImage for the CPU or ID3D11DeviceContext->CopyResource for the GPU. In my opinion you cant expect developers to write their own non-temporal copy functions in Assembler, nor that developers start learning DirectX Math or OpenCL to write their own conversion functions. I am aware that my app is not the usual kind of program with its hooking but CopyResource definitly screams for a conversion parameter.

    The games i tested so far mostly have the same D3DFORMAT in Direct3D 9 mode and working fine with MFCreateDXSurfaceBuffer, but now i have to start testing out all the possible formats to make sure it still works with the other formats too. If only ARGB and BGRA ( big little endian ) formats are working with MFCreateDXSurfaceBuffer and MFCreateDXGISurfaceBuffer then its useless to me and i have to rely on my other approaches which are locking and mapping. Dont get me wrong, they are also working but as meantioned in my other thread in this forum,  the bit shifting slows down the fps of a game about 75 % when encoding with media foundation ! With the non-temporal MFCopyImage function it slows down only 15-20 % but the colors are wrong !

    I have to continue researching this matter now because loosing 75 % fps when playing is unacceptable. I will keep you updated in my other thread if i find a solution.

    regards

    coOKie




    • Marked as answer by Francis Grave Friday, November 15, 2013 1:20 AM
    • Unmarked as answer by Francis Grave Friday, November 15, 2013 9:50 AM
    • Edited by Francis Grave Friday, November 15, 2013 2:04 PM edit
    • Marked as answer by Francis Grave Wednesday, November 27, 2013 12:44 AM
    Friday, November 15, 2013 1:15 AM

All replies

  • Well after testing a BGRA Format now the function succeeds...

    Now heres my problem :

    I am not the owner of the source texture. I am hooking games and extracting their backbuffers and so i have no control over the creation parameters of the textures.

    One way i see is to build up a 2nd render target and set it via OMSetRenderTargets(). Would it slow down the fps significantly when i render to 2 different targets ? Well i will see that tomorrow when i start testing this out.

    If someone at Microsoft reads this, please we need format conversions in either MFCopyImage for the CPU or ID3D11DeviceContext->CopyResource for the GPU. In my opinion you cant expect developers to write their own non-temporal copy functions in Assembler, nor that developers start learning DirectX Math or OpenCL to write their own conversion functions. I am aware that my app is not the usual kind of program with its hooking but CopyResource definitly screams for a conversion parameter.

    The games i tested so far mostly have the same D3DFORMAT in Direct3D 9 mode and working fine with MFCreateDXSurfaceBuffer, but now i have to start testing out all the possible formats to make sure it still works with the other formats too. If only ARGB and BGRA ( big little endian ) formats are working with MFCreateDXSurfaceBuffer and MFCreateDXGISurfaceBuffer then its useless to me and i have to rely on my other approaches which are locking and mapping. Dont get me wrong, they are also working but as meantioned in my other thread in this forum,  the bit shifting slows down the fps of a game about 75 % when encoding with media foundation ! With the non-temporal MFCopyImage function it slows down only 15-20 % but the colors are wrong !

    I have to continue researching this matter now because loosing 75 % fps when playing is unacceptable. I will keep you updated in my other thread if i find a solution.

    regards

    coOKie




    • Marked as answer by Francis Grave Friday, November 15, 2013 1:20 AM
    • Unmarked as answer by Francis Grave Friday, November 15, 2013 9:50 AM
    • Edited by Francis Grave Friday, November 15, 2013 2:04 PM edit
    • Marked as answer by Francis Grave Wednesday, November 27, 2013 12:44 AM
    Friday, November 15, 2013 1:15 AM
  • That one particularly game has DXGI_FORMAT_R8G8B8A8_UNORM format on its backbuffer, so i tested the following formats which are in the "compatible" group :

    DXGI_FORMAT_R8G8B8A8_TYPELESS
    DXGI_FORMAT_R8G8B8A8_UNORM
    DXGI_FORMAT_R8G8B8A8_UNORM_SRGB
    DXGI_FORMAT_R8G8B8A8_UINT
    DXGI_FORMAT_R8G8B8A8_SNORM
    DXGI_FORMAT_R8G8B8A8_SINT

    From these formats only the SRGB type is accepted by MFCreateDXGISurfaceBuffer, but i think thats just something they forgot to build into the routine and was never meantioned to be the only RGBA 32 format working with that function. If i am wrong and it is the only RGBA format working, how can i convert from the UNORM to the SRGB type ?

    regards

    coOKie


    Friday, November 15, 2013 10:05 AM
  • After a lot of testing i found out that the RGBA / ABGR ( big little endian ) formats are not supported by MFCreateDXSurfaceBuffer and MFCreateDXGISurfaceBuffer.

    Like in DXGI_FORMAT in Direct3D 9 there are restrictions which formats can be backbuffer or not and all those who supporting it are also supported by MFCreateDXSurfaceBuffer. This makes 100 % sure all backbuffer types can be encoded using MFCreateDXSurfaceBuffer.

    For Direct3D 10 and 11 with MFCreateDXGISurfaceBuffer...well...where even to begin with. Lets stay with RGBA Problem here on topic of the thread to limit the problems of this function. A lot of games out there have RGBA DXGI format and thats a format like nowhere supported in Media Foundation. MFCreateDXGISurfaceBuffer declines it, and there are no conversion functions in Media Foundation, not even in Direct3D 10 or 11.

    I read about the D3DX_DXGIFormatConvert.inl inline conversions, and they might be handy somehow but still, D3DX is deprecated to say at first ( even though it could be easiely rewritten in DirectX Math ), and secondly i hook games and extracting their backbuffer so i have no control over the creation parameters to bind the buffer as shader resource. However i do see the possibility to create my own rendertarget and bind it as shader resource, then use CopyResource and from my rendertarget i create a Unordered Access View and then render it to a 3rd texture which has ARGB / BGRA format.

    And another way could be image processing like with DXVA, but i have to continue my research on that one. It would only work if you can have D3D surfaces as Input.

    regards

    coOKie




    Sunday, November 17, 2013 6:34 PM
  • FWIW, DirectXTex provides CPU-based conversion using DirectXMath of all the DXGI color formats. It doesn't currently support the various video formats.

    Sunday, November 17, 2013 11:44 PM
  • Thanks for your reply Chuck,

    but i knew that and this library is no option for me as it is not using "non-temporal" write operations for storage. As in my other threads meantioned i am live encoding the backbuffer from games and this is performance critical, and cpu conversion alone is to slow ( even if it is SSE optimized ). I am able to write conversions myself with DirectX Math, my problem is that it is not fast enough for live encoding.

    The only way to remain enough speed (fps) when live encoding a backbuffer with RGBA / ABGR format is when you use MFCopyImage which is described as "non-temporal" SSE2 optimized copy function. If u read my other threads you can clearly see the difference in performance with and without this store function.

    The problem is that this function cannot perform any type of conversion itself  and so the colors are wrong. Any kind of format conversion with the CPU before this function slows down the game by 75 % ! Conversion is not the problem itself, its rather about conversion speed i am struggling with. If you omit format conversion before MFCopyImage it only slows down the game 10-20 % in fps. My thought was that i could modify this non-temporal function and write my own custom function which handles conversion and storage at once, but the problem is that i would need to digg into Assembler or learn to use SSE2 instructions.

    The fastest solution for format conversion should be using the GPU directly and thats what i am investigating now.

    regards

    coOKie


    Monday, November 18, 2013 2:59 AM