locked
NV12 texture display problem

    Question

  • Hello, I'm trying to play video frames to a texture. The video is encoded in h264 and I'm using the Media Foundation API to output the data to NV12 format. Basically my problem is a continuation of what JanChen had a few months ago Using NV12 texture in Windows Developer Preview . I create two shader resource views - one of type DXGI_FORMAT_R8_UNORM and the other of DXGI_FORMAT_R8G8_UNORM

    if(DXGI_FORMAT_NV12 == videoFormat)
    	{
    		D3D11_SHADER_RESOURCE_VIEW_DESC srv;
    		ZeroMemory(&srv, sizeof(D3D11_SHADER_RESOURCE_VIEW_DESC));
    		srv.Format = DXGI_FORMAT_R8_UNORM;
    		srv.ViewDimension = D3D11_SRV_DIMENSION_TEXTURE2D;
    		srv.Texture2D.MipLevels = 1;
    		srv.Texture2D.MostDetailedMip = 0;
    		HRESULT hr = device->CreateShaderResourceView(texture, &srv, &textureView1);
    
    		if(FAILED(hr))
    		{
    			close_all();
    			throw std::runtime_error("Failed to the first create shader resource view");
    		}
    
    		srv.Format = DXGI_FORMAT_R8G8_UNORM;
    		hr = device->CreateShaderResourceView(texture, &srv, &textureView2);
    		
    		if(FAILED(hr))
    		{
    			close_all();
    			throw std::runtime_error("Failed to the second create shader resource view");
    		}

    I set both texture views to the PixelShader with PSSetShaderResources() and in my pixel shader I have

    Texture2D<float> tex2d1 : register(t0);
    Texture2D<float2> tex2d2 : register(t1);
    
    SamplerState samp : register(s0);
    
    
    struct PS_INPUT
    {
    	float4 position : SV_Position;
    	float2 texcoord : TEXCOORD;
    };
    
    float4 main(PS_INPUT input) : SV_TARGET
    {
    
    	float r = tex2d1.Sample(samp, input.texcoord);
    	float2 gb = tex2d2.Sample(samp, input.texcoord);
    	return float4(r, gb.x, gb.y, 1.0);
    }

    and it gives this output where the colors are messed up:

    Can anyone help, please?



    • Edited by Brianmij Tuesday, August 07, 2012 4:13 PM
    Tuesday, August 07, 2012 4:11 PM

Answers

All replies

  • Hello,

     

    Thanks for your feedback, I will involve more experts to investigate it.

     

    Best regards,

    Jesse


    Jesse Jiang [MSFT]
    MSDN Community Support | Feedback to us

    Friday, August 10, 2012 7:30 AM
    Moderator
  • Hello,

    Please take a look at the thread listed below. If the answer that Kenny offered doesn't work for you please let me know.

    http://social.msdn.microsoft.com/Forums/en-US/winappswithnativecode/thread/96117e0f-3f6a-4d37-b374-b3d081c3d71e/

    -James


    Windows Media SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

    Friday, August 10, 2012 10:56 PM
    Moderator
  • Yes! That is it. h.264 to RGB32 works perfectly. IMFSourceReaderEx is sweet! Thanks you, James.
    Tuesday, August 14, 2012 4:04 AM
  • Surely DXGI_FORMAT_R8G8_UNORM should be DXGI_FORMAT_R8G8B8_UNORM because you are losing the blue colour.

    Why do you need 2 shaders ? I just use one texture and one shader.


    n.Wright


    Saturday, October 12, 2013 10:10 PM
  • You cannot create Direct3D 11 shader resource views directly for video formats. The Direct3D resource is a DXGI_FORMAT_NV12, which is a YUV format. In order to process it in Direct3D 11, you create two Shader Resource Views on the same Direct3D resource. The DXGI_FORMAT_R8_UNORM view is the Y channel. The DXGI_FORMAT_R8G8_UNORM is the UV channels. Remember that in many video formats, the physical pixel size of the Y channel is double the size of the U and V channels.

    The OP problem is that the return float4(r, gb.x, gb.y, 1.0); statement is still in the YUV colorspace. He did not convert the data to RGB which is what is expected to come back from the pixel shader.

    See MSDN for details.



    Monday, October 14, 2013 4:59 PM