locked
How to get pixels from SurfaceImageSource for processing?

    Question

  • I am able to use Direct2D to draw stuff on a XAML rectangle using the SurfaceImageSource approach described in http://msdn.microsoft.com/en-us/library/windows/apps/hh825871.aspx.

    Now I want to get the individual pixels in preferably RGBA format so that I can further process them with CPU, and eventually create a normal WriteableBitmap object.

    I tried something like this...

        D2D1_SIZE_U bmpSize;
    
        bmpSize.width = 400;
        bmpSize.height = 300;
    
        ID2D1Bitmap *bmp;
        D2D1_BITMAP_PROPERTIES bmpProps;
        ZeroMemory(&bmpProps,sizeof(bmpProps));
        D2D1_PIXEL_FORMAT format;
    
        format.format = DXGI_FORMAT_UNKNOWN;
        format.alphaMode = D2D1_ALPHA_MODE_PREMULTIPLIED;
    
        bmpProps = D2D1::BitmapProperties(format, dpiX, dpiY);
        m_d2dRenderTarget->CreateBitmap(bmpSize, bmpProps, &bmp);

    But it just crashes and I am not sure what is the right way to do so. 




    • Edited by pragos Wednesday, July 25, 2012 3:55 AM
    Wednesday, July 25, 2012 3:50 AM

Answers

  • Actually I found one solution to get the pixels into my own memory buffer. I create another bitmap with D2D1_BITMAP_OPTIONS_CPU_READ and copy the old bitmap into the new one using CopyFromBitmap method. Since the new bitmap is created with CPU read option, I can now use the Map function to get the pointer to it's pixel data, which I then copy to my own memory location.

    I do not use surface image to render anymore for this project. My original render target is another D2D1Bitmap1, from which I copy.

    D2D1_BITMAP_PROPERTIES1 prop = D2D1::BitmapProperties1(
    		D2D1_BITMAP_OPTIONS_CPU_READ | D2D1_BITMAP_OPTIONS_CANNOT_DRAW,
    		D2D1::PixelFormat(
    		DXGI_FORMAT_B8G8R8A8_UNORM,
    		D2D1_ALPHA_MODE_PREMULTIPLIED
    		)
    		);
    
    	ComPtr<ID2D1Bitmap1> otherBmp;
    	m_d2d1DeviceContext->CreateBitmap(
    		size, 
    		nullptr,
    		0,
    		&prop,
    		&otherBmp
    		);
    
    	D2D1_RECT_U rect = {0, 0, W, H};
    	D2D1_POINT_2U point = {0, 0};
    	hr = otherBmp->CopyFromBitmap(&point, m_d2d1TargetBitmap.Get(), &rect);  
    	
    	D2D1_MAP_OPTIONS options = D2D1_MAP_OPTIONS_READ;
    	D2D1_MAPPED_RECT mappedRect;
    	hr = otherBmp->Map(options, &mappedRect);
    	memcpy(myPixelBuffer, mappedRect.bits, W * H * 4);
    	otherBmp->Unmap();

    • Marked as answer by Jesse Jiang Thursday, August 02, 2012 7:15 AM
    Tuesday, July 31, 2012 8:55 AM

All replies

  • Is it even possible to get the pixels from SurfaceImageSource?

    
    
    	m_d2dFactory->CreateDevice(dxgiDevice.Get(), &m_d2d1Device);
    	m_d2d1Device->CreateDeviceContext(D2D1_DEVICE_CONTEXT_OPTIONS_NONE, &m_d2d1DeviceContext);
    
    			Microsoft::WRL::ComPtr<ID2D1Bitmap1> bitmap;
    			m_d2d1DeviceContext->CreateBitmapFromDxgiSurface(surface, NULL, &bitmap);
    			m_d2d1DeviceContext->SetTarget(bitmap.Get());
    
    			m_d2d1DeviceContext->BeginDraw();
    			ID2D1SolidColorBrush    *pBrush;
    			const D2D1_COLOR_F color = D2D1::ColorF(1.0f, 1.0f, 0);
                hr = m_d2d1DeviceContext->CreateSolidColorBrush(color, &pBrush);
    			m_d2d1DeviceContext->Clear( D2D1::ColorF(D2D1::ColorF::SkyBlue) );
    			m_d2d1DeviceContext->FillEllipse(ellipse, pBrush);
    			m_d2d1DeviceContext->EndDraw();
    

    If I use this approach I do get a ID2D1Bitmap1 but I don't seem to be able to get pixels from it. I read that I can D1D1Bitmap1::Map method to map the pixels mapped to CPU readable memory. But for that the I need to create the bitmap with the CPU_READ option which is not possible in this approach.

    Wednesday, July 25, 2012 6:44 AM
  • Hi,

    Maybe you can try to use WIC to custom image. Please follow these sample codes
    http://code.msdn.microsoft.com/windowsapps/SaveAsImageFile-68073cb0
    http://code.msdn.microsoft.com/windowsapps/Direct2D-custom-image-7a1f2cb5

    Best regards,
    Jesse


    Jesse Jiang [MSFT]
    MSDN Community Support | Feedback to us

    • Proposed as answer by Jesse Jiang Monday, July 30, 2012 7:11 AM
    Friday, July 27, 2012 7:49 AM
  • Actually I found one solution to get the pixels into my own memory buffer. I create another bitmap with D2D1_BITMAP_OPTIONS_CPU_READ and copy the old bitmap into the new one using CopyFromBitmap method. Since the new bitmap is created with CPU read option, I can now use the Map function to get the pointer to it's pixel data, which I then copy to my own memory location.

    I do not use surface image to render anymore for this project. My original render target is another D2D1Bitmap1, from which I copy.

    D2D1_BITMAP_PROPERTIES1 prop = D2D1::BitmapProperties1(
    		D2D1_BITMAP_OPTIONS_CPU_READ | D2D1_BITMAP_OPTIONS_CANNOT_DRAW,
    		D2D1::PixelFormat(
    		DXGI_FORMAT_B8G8R8A8_UNORM,
    		D2D1_ALPHA_MODE_PREMULTIPLIED
    		)
    		);
    
    	ComPtr<ID2D1Bitmap1> otherBmp;
    	m_d2d1DeviceContext->CreateBitmap(
    		size, 
    		nullptr,
    		0,
    		&prop,
    		&otherBmp
    		);
    
    	D2D1_RECT_U rect = {0, 0, W, H};
    	D2D1_POINT_2U point = {0, 0};
    	hr = otherBmp->CopyFromBitmap(&point, m_d2d1TargetBitmap.Get(), &rect);  
    	
    	D2D1_MAP_OPTIONS options = D2D1_MAP_OPTIONS_READ;
    	D2D1_MAPPED_RECT mappedRect;
    	hr = otherBmp->Map(options, &mappedRect);
    	memcpy(myPixelBuffer, mappedRect.bits, W * H * 4);
    	otherBmp->Unmap();

    • Marked as answer by Jesse Jiang Thursday, August 02, 2012 7:15 AM
    Tuesday, July 31, 2012 8:55 AM
  • I tried this and all the buffer is zero's! did i miss anything? any one tried it and works for him?
    Saturday, August 25, 2012 6:42 PM
  • Sam, what is your render target? I could not get the pixel buffer from SufraceImage render target. But if I have my render target as D2D1Bitmap1, this solution works. First you need to draw something on the target bitmap, which is "m_d2d1TargetBitmap" in this case, and then you can use this code to get the pixels from that target. The device context here is associated with m_d2d1TargetBitmap.
    Sunday, August 26, 2012 1:58 PM
  • Yes, I Set the target and then drew some rectangle and then try to save the target bitmap, but its 0x00000000

    here is what I did

     m_d2dContext->SetTarget(m_d2dTargetBitmap.Get());
     Windows::Foundation::Rect rect2 = Windows::Foundation::Rect(0,0,800,800);
     
     FillSolidRect(Windows::UI::Colors::Red, rect2);

    D2D1_BITMAP_PROPERTIES1 prop = D2D1::BitmapProperties1(
     D2D1_BITMAP_OPTIONS_CPU_READ | D2D1_BITMAP_OPTIONS_CANNOT_DRAW,
      D2D1::PixelFormat(
      DXGI_FORMAT_B8G8R8A8_UNORM,
      D2D1_ALPHA_MODE_PREMULTIPLIED
      )
      );
    D2D1_SIZE_U pixelSize = m_d2dContext->GetPixelSize();
    D2D1_SIZE_U size = {800,800};
     ComPtr<ID2D1Bitmap1> otherBmp;
     m_d2dContext->CreateBitmap(
      pixelSize,
      nullptr,
      pixelSize.width * 4,
      &prop,
      &otherBmp
      );

     D2D1_RECT_U rect = {0, 0, 800, 800};
     D2D1_POINT_2U point = {0, 0};
     HRESULT hr = otherBmp->CopyFromBitmap(&point, m_d2dTargetBitmap.Get(), &rect); 
     Clear(Windows::UI::Colors::Beige);
     FillSolidRect(Windows::UI::Colors::Yellow, rect2);
     //m_d2dContext->DrawBitmap(otherBmp.Get());
     D2D1_MAP_OPTIONS options = D2D1_MAP_OPTIONS_READ;
     D2D1_MAPPED_RECT mappedRect;
     hr = otherBmp->Map(options, &mappedRect);
     
     byte myPixelBuffer[50];
     memcpy(myPixelBuffer, mappedRect.bits, 50);
     otherBmp->Unmap();

     return;

    Monday, August 27, 2012 2:33 PM
  • Not sure what might be wrong, I am very new to windows. Here is my code that is working, if that helps...

    Initialization code...

    void MainPage::InitializeDevice() { D2D1_FACTORY_OPTIONS options; ZeroMemory(&options, sizeof(D2D1_FACTORY_OPTIONS)); #if defined(_DEBUG) // If the project is in a debug build, enable Direct2D debugging via SDK Layers options.debugLevel = D2D1_DEBUG_LEVEL_INFORMATION; #endif ThrowIfFailed(D2D1CreateFactory( D2D1_FACTORY_TYPE_SINGLE_THREADED, __uuidof(ID2D1Factory1), &options, &m_d2dFactory )); UINT creationFlags = D3D11_CREATE_DEVICE_BGRA_SUPPORT; #if defined(_DEBUG) // If the project is in a debug build, enable debugging via SDK Layers with this flag. creationFlags |= D3D11_CREATE_DEVICE_DEBUG; #endif D3D_FEATURE_LEVEL featureLevels[] = { D3D_FEATURE_LEVEL_11_1, D3D_FEATURE_LEVEL_11_0, D3D_FEATURE_LEVEL_10_1, D3D_FEATURE_LEVEL_10_0, D3D_FEATURE_LEVEL_9_3, D3D_FEATURE_LEVEL_9_2, D3D_FEATURE_LEVEL_9_1 }; // ... HRESULT hr = D3D11CreateDevice( NULL, D3D_DRIVER_TYPE_HARDWARE, NULL, creationFlags, featureLevels, ARRAYSIZE(featureLevels), D3D11_SDK_VERSION, &m_d3dDevice, NULL, &m_d3dContext); Microsoft::WRL::ComPtr<IDXGIDevice> dxgiDevice; m_d3dDevice.As(&dxgiDevice); m_d2dFactory->CreateDevice(dxgiDevice.Get(), &m_d2d1Device);

    m_d2d1Device->CreateDeviceContext(D2D1_DEVICE_CONTEXT_OPTIONS_NONE, &m_d2d1DeviceContext); }

    Drawing code...

        D2D1_BITMAP_PROPERTIES1 renderTargetProperties = D2D1::BitmapProperties1(
    		D2D1_BITMAP_OPTIONS_TARGET,
    		D2D1::PixelFormat(
    		DXGI_FORMAT_B8G8R8A8_UNORM,
    		D2D1_ALPHA_MODE_PREMULTIPLIED
    		)
    		);
    
        D2D1_SIZE_U size = {3264, 2448};
        m_d2d1DeviceContext->CreateBitmap(
    		size, 
    		nullptr,
    		0,
    		&renderTargetProperties,
    		&m_d2d1TargetBitmap
    		);
    
        m_d2d1DeviceContext->SetTarget(m_d2d1TargetBitmap.Get());
    
        m_d2d1DeviceContext->BeginDraw();
        m_d2d1DeviceContext->Clear( D2D1::ColorF(D2D1::ColorF::Black) );
        m_d2d1DeviceContext->FillEllipse(ellipse, pBrush);
        m_d2d1DeviceContext->EndDraw();
    
    Pixel read code...

    	D2D1_BITMAP_PROPERTIES1 prop1 = D2D1::BitmapProperties1(
    		D2D1_BITMAP_OPTIONS_CPU_READ | D2D1_BITMAP_OPTIONS_CANNOT_DRAW,
    		D2D1::PixelFormat(
    		DXGI_FORMAT_B8G8R8A8_UNORM,
    		D2D1_ALPHA_MODE_PREMULTIPLIED
    		)
    		);
    
        ComPtr<ID2D1Bitmap1> otherBmp;
        m_d2d1DeviceContext->CreateBitmap(
    		size, 
    		nullptr,
    		0,
    		&prop1,
    		&otherBmp
    		);
    
        D2D1_RECT_U rect = {0, 0, 3264, 2448};
        D2D1_POINT_2U point = {0, 0};
        hr = otherBmp->CopyFromBitmap(&point, m_d2d1TargetBitmap.Get(), &rect);  
    	
        D2D1_MAP_OPTIONS options = D2D1_MAP_OPTIONS_READ;
        D2D1_MAPPED_RECT mappedRect;
        hr = otherBmp->Map(options, &mappedRect);
        memcpy(pixels, mappedRect.bits, W * H * 4);
        otherBmp->Unmap();
    


    Tuesday, August 28, 2012 9:44 AM
  • Thanks. I resolve my issue using WicBitmap and then create RenderTarget from Wic.
    Tuesday, August 28, 2012 4:03 PM
  • Pragos answered worked for me, but I figured I'd post this for anyone else stumbling across it. The pitch value of the mapped rect isn't guaranteed to be the same as the width * 4. I mean, it might always work out to be the same, but the existence of the pitch member to D2D_MAPPED_RECT implies that it isn't. So this code should be safe, in all cases:

    for (int y = 0; y < H; y++)

    {

        // assuming pixels is a char*

        memcpy(pixels + (y * W * 4), mappedRect.bits + mappedRect.pitch, W * 4);

    }

    Friday, March 06, 2015 6:56 PM