locked
Using the Media Engine in Win8 with different texture formats

    Question

  • Hi.

    I have been trying to use Win8's Media Engine to play a movie to a texture.

    I was told that the Media Engine should be capable of outputting in just about any format.  Anytime I try changing the format from anything but the default DXGI_FORMAT_B8G8R8A8_UNORM I get a COM exception 0xC00D36B4 (MF_E_INVALIDMEDIATYPE)

    So for example, if I use the code from the MSDN sample: "Media engine native C++ video playback sample" and change the format used from DXGI_FORMAT_B8G8R8A8_UNORM to say DXGI_FORMAT_R8G8B8A8_UNORM it barfs during the call to TransferVideoFrame with the mentioned Com exception.

    In my real world case, I have a texture 2D (ID3D11Texture2D) passed to me from Unity in DXGI_FORMAT_R8G8B8A8_UNORM and I only seem able to get the media engine working with DXGI_FORMAT_B8G8R8A8_UNORM so I dont know how to get the data from one texture2D to another while performing the color space conversion.  I imaging there is some way using a shader, but before I attempt learning shaders, I want to confirm if there isnt some automatic way to get this working.

    Any advice will be appreciated.

    Cheers.

    Sean

    [UPDATE] So I started looking at the source reader examples and I ran into the same problem as soon as I tried calling SetCurrentMediaType with a RGB format : MF_E_INVALIDMEDIATYPE

    I finally figured out that if I add the attribute: MF_SOURCE_READER_ENABLE_ADVANCED_VIDEO_PROCESSING to the source reader, the error finally disappears.

    So it would seem that somewhere inside of the default behavior of the media engine, it doesn't setup this attribute.  Can this be setup somehow or am I stuck using the source reader. The media engine is so easy to use that I would prefer using it. If I have to continue using the source reader, I assume I can use the Map call to copy the frame data from a IMFMediaBuffer to the ID3D11Texture2D?  It seems like I will have to handle the audio myself also.  Is there a good example on how to play a movie using the source reader?

    Again, any advice will be appreciated.

    Cheers.

    Sean

    [UPDATE] Made some more progress with the media player. I figured out that I could use an intermediate texture to get the frame image copied into the Unity texture. The final result needs to have its colorspace modified, but its working.

    Essentially I do this:

    m_spMediaEngine->TransferVideoFrame( m_SourceTexture.Get(), &rect, &rcTarget, &m_bkgColor ) );

    where m_SourceTexture is format DXGI_FORMAT_B8G8R8A8_UNORM and has the flags: D3D11_BIND_SHADER_RESOURCE | D3D11_BIND_RENDER_TARGET

    then I call:

    pContext->CopyResource( m_IntermediateTexture.Get(), m_SourceTexture.Get() );

    where m_IntermediateTexture is format DXGI_FORMAT_B8G8R8A8_UNORM and has the flags: D3D11_USAGE_STAGING & D3D11_CPU_ACCESS_READ

    Then I call:

    pContext->Map( m_IntermediateTexture.Get(), 0, D3D11_MAP_READ, 0, &mappedSourceResource ) );
    BYTE* sourceData = reinterpret_cast<BYTE*>( mappedSourceResource.pData );

    And finally:

    pContext->UpdateSubresource( m_TargetTexture.Get(), 0, NULL, sourceData, mappedSourceResource.RowPitch, 0 );
    pContext->Unmap( m_IntermediateTexture.Get(), 0 );
     

    Since the target texture that Unity gave me is DXGI_FORMAT_R8G8B8A8_UNORM, the colorspace is messed up but I think I can fix that with a shader in Unity.

    Regardless, this seems a terribly convoluted way to copy textures.

    Is there some reason why the MediaEngine cant output in any other format than DXGI_FORMAT_B8G8R8A8_UNORM?  If I could get the source and target textures the same format, I could perform a single copy resource?

    Could all of this copying be improved with a compute shader or something that would copy from GPU to GPU.

    Again, any advice will be appreciated.

    Sean



    Saturday, June 22, 2013 8:04 PM