How to get pixels from SurfaceImageSource for processing?

Question

I am able to use Direct2D to draw stuff on a XAML rectangle using the SurfaceImageSource approach described in http://msdn.microsoft.com/en-us/library/windows/apps/hh825871.aspx.

Now I want to get the individual pixels in preferably RGBA format so that I can further process them with CPU, and eventually create a normal WriteableBitmap object.

I tried something like this...

    D2D1_SIZE_U bmpSize;

    bmpSize.width = 400;
    bmpSize.height = 300;

    ID2D1Bitmap *bmp;
    D2D1_BITMAP_PROPERTIES bmpProps;
    ZeroMemory(&bmpProps,sizeof(bmpProps));
    D2D1_PIXEL_FORMAT format;

    format.format = DXGI_FORMAT_UNKNOWN;
    format.alphaMode = D2D1_ALPHA_MODE_PREMULTIPLIED;

    bmpProps = D2D1::BitmapProperties(format, dpiX, dpiY);
    m_d2dRenderTarget->CreateBitmap(bmpSize, bmpProps, &bmp);

But it just crashes and I am not sure what is the right way to do so. 

Answer 1

Is it even possible to get the pixels from SurfaceImageSource?

	m_d2dFactory->CreateDevice(dxgiDevice.Get(), &m_d2d1Device);
	m_d2d1Device->CreateDeviceContext(D2D1_DEVICE_CONTEXT_OPTIONS_NONE, &m_d2d1DeviceContext);

			Microsoft::WRL::ComPtr<ID2D1Bitmap1> bitmap;
			m_d2d1DeviceContext->CreateBitmapFromDxgiSurface(surface, NULL, &bitmap);
			m_d2d1DeviceContext->SetTarget(bitmap.Get());

			m_d2d1DeviceContext->BeginDraw();
			ID2D1SolidColorBrush    *pBrush;
			const D2D1_COLOR_F color = D2D1::ColorF(1.0f, 1.0f, 0);
            hr = m_d2d1DeviceContext->CreateSolidColorBrush(color, &pBrush);
			m_d2d1DeviceContext->Clear( D2D1::ColorF(D2D1::ColorF::SkyBlue) );
			m_d2d1DeviceContext->FillEllipse(ellipse, pBrush);
			m_d2d1DeviceContext->EndDraw();

If I use this approach I do get a ID2D1Bitmap1 but I don't seem to be able to get pixels from it. I read that I can D1D1Bitmap1::Map method to map the pixels mapped to CPU readable memory. But for that the I need to create the bitmap with the CPU_READ option which is not possible in this approach.

Answer 2

Hi,

Maybe you can try to use WIC to custom image. Please follow these sample codes
http://code.msdn.microsoft.com/windowsapps/SaveAsImageFile-68073cb0
http://code.msdn.microsoft.com/windowsapps/Direct2D-custom-image-7a1f2cb5

Best regards,
Jesse

---

Jesse Jiang [MSFT]
MSDN Community Support | Feedback to us

Answer 3

Actually I found one solution to get the pixels into my own memory buffer. I create another bitmap with D2D1_BITMAP_OPTIONS_CPU_READ and copy the old bitmap into the new one using CopyFromBitmap method. Since the new bitmap is created with CPU read option, I can now use the Map function to get the pointer to it's pixel data, which I then copy to my own memory location.

I do not use surface image to render anymore for this project. My original render target is another D2D1Bitmap1, from which I copy.

D2D1_BITMAP_PROPERTIES1 prop = D2D1::BitmapProperties1(
		D2D1_BITMAP_OPTIONS_CPU_READ | D2D1_BITMAP_OPTIONS_CANNOT_DRAW,
		D2D1::PixelFormat(
		DXGI_FORMAT_B8G8R8A8_UNORM,
		D2D1_ALPHA_MODE_PREMULTIPLIED
		)
		);

	ComPtr<ID2D1Bitmap1> otherBmp;
	m_d2d1DeviceContext->CreateBitmap(
		size, 
		nullptr,
		0,
		&prop,
		&otherBmp
		);

	D2D1_RECT_U rect = {0, 0, W, H};
	D2D1_POINT_2U point = {0, 0};
	hr = otherBmp->CopyFromBitmap(&point, m_d2d1TargetBitmap.Get(), &rect);  
	
	D2D1_MAP_OPTIONS options = D2D1_MAP_OPTIONS_READ;
	D2D1_MAPPED_RECT mappedRect;
	hr = otherBmp->Map(options, &mappedRect);
	memcpy(myPixelBuffer, mappedRect.bits, W * H * 4);
	otherBmp->Unmap();

Answer 4

I tried this and all the buffer is zero's! did i miss anything? any one tried it and works for him?

Answer 5

Sam, what is your render target? I could not get the pixel buffer from SufraceImage render target. But if I have my render target as D2D1Bitmap1, this solution works. First you need to draw something on the target bitmap, which is "m_d2d1TargetBitmap" in this case, and then you can use this code to get the pixels from that target. The device context here is associated with m_d2d1TargetBitmap.

Answer 6

Yes, I Set the target and then drew some rectangle and then try to save the target bitmap, but its 0x00000000

here is what I did

 m_d2dContext->SetTarget(m_d2dTargetBitmap.Get());
 Windows::Foundation::Rect rect2 = Windows::Foundation::Rect(0,0,800,800);
 
 FillSolidRect(Windows::UI::Colors::Red, rect2);

D2D1_BITMAP_PROPERTIES1 prop = D2D1::BitmapProperties1(
 D2D1_BITMAP_OPTIONS_CPU_READ | D2D1_BITMAP_OPTIONS_CANNOT_DRAW,
  D2D1::PixelFormat(
  DXGI_FORMAT_B8G8R8A8_UNORM,
  D2D1_ALPHA_MODE_PREMULTIPLIED
  )
  );
D2D1_SIZE_U pixelSize = m_d2dContext->GetPixelSize();
D2D1_SIZE_U size = {800,800};
 ComPtr<ID2D1Bitmap1> otherBmp;
 m_d2dContext->CreateBitmap(
  pixelSize,
  nullptr,
  pixelSize.width * 4,
  &prop,
  &otherBmp
  );

 D2D1_RECT_U rect = {0, 0, 800, 800};
 D2D1_POINT_2U point = {0, 0};
 HRESULT hr = otherBmp->CopyFromBitmap(&point, m_d2dTargetBitmap.Get(), &rect); 
 Clear(Windows::UI::Colors::Beige);
 FillSolidRect(Windows::UI::Colors::Yellow, rect2);
 //m_d2dContext->DrawBitmap(otherBmp.Get());
 D2D1_MAP_OPTIONS options = D2D1_MAP_OPTIONS_READ;
 D2D1_MAPPED_RECT mappedRect;
 hr = otherBmp->Map(options, &mappedRect);
 
 byte myPixelBuffer[50];
 memcpy(myPixelBuffer, mappedRect.bits, 50);
 otherBmp->Unmap();

 return;

Answer 7

Not sure what might be wrong, I am very new to windows. Here is my code that is working, if that helps...

Initialization code...

void MainPage::InitializeDevice()
{
    D2D1_FACTORY_OPTIONS options;
    ZeroMemory(&options, sizeof(D2D1_FACTORY_OPTIONS));
#if defined(_DEBUG)
     // If the project is in a debug build, enable Direct2D debugging via SDK Layers
    options.debugLevel = D2D1_DEBUG_LEVEL_INFORMATION;
#endif

     ThrowIfFailed(D2D1CreateFactory(
            D2D1_FACTORY_TYPE_SINGLE_THREADED,
            __uuidof(ID2D1Factory1),
            &options,
            &m_d2dFactory
            ));

    UINT creationFlags = D3D11_CREATE_DEVICE_BGRA_SUPPORT;

#if defined(_DEBUG)
    // If the project is in a debug build, enable debugging via SDK Layers with this flag.
    creationFlags |= D3D11_CREATE_DEVICE_DEBUG;
#endif

    D3D_FEATURE_LEVEL featureLevels[] = 
	{
		D3D_FEATURE_LEVEL_11_1,
		D3D_FEATURE_LEVEL_11_0,
		D3D_FEATURE_LEVEL_10_1,
		D3D_FEATURE_LEVEL_10_0,
		D3D_FEATURE_LEVEL_9_3,
		D3D_FEATURE_LEVEL_9_2,
		D3D_FEATURE_LEVEL_9_1
	};
	// ...
    HRESULT hr = D3D11CreateDevice(
		NULL,
		D3D_DRIVER_TYPE_HARDWARE,
		NULL,
		creationFlags,
		featureLevels,
		ARRAYSIZE(featureLevels),
		D3D11_SDK_VERSION,
		&m_d3dDevice,
		NULL,
		&m_d3dContext);

    Microsoft::WRL::ComPtr<IDXGIDevice> dxgiDevice;
    m_d3dDevice.As(&dxgiDevice);

    m_d2dFactory->CreateDevice(dxgiDevice.Get(), &m_d2d1Device);
    m_d2d1Device->CreateDeviceContext(D2D1_DEVICE_CONTEXT_OPTIONS_NONE, &m_d2d1DeviceContext);
}

Drawing code...

    D2D1_BITMAP_PROPERTIES1 renderTargetProperties = D2D1::BitmapProperties1(
		D2D1_BITMAP_OPTIONS_TARGET,
		D2D1::PixelFormat(
		DXGI_FORMAT_B8G8R8A8_UNORM,
		D2D1_ALPHA_MODE_PREMULTIPLIED
		)
		);

    D2D1_SIZE_U size = {3264, 2448};
    m_d2d1DeviceContext->CreateBitmap(
		size, 
		nullptr,
		0,
		&renderTargetProperties,
		&m_d2d1TargetBitmap
		);

    m_d2d1DeviceContext->SetTarget(m_d2d1TargetBitmap.Get());

    m_d2d1DeviceContext->BeginDraw();
    m_d2d1DeviceContext->Clear( D2D1::ColorF(D2D1::ColorF::Black) );
    m_d2d1DeviceContext->FillEllipse(ellipse, pBrush);
    m_d2d1DeviceContext->EndDraw();

Pixel read code...

	D2D1_BITMAP_PROPERTIES1 prop1 = D2D1::BitmapProperties1(
		D2D1_BITMAP_OPTIONS_CPU_READ | D2D1_BITMAP_OPTIONS_CANNOT_DRAW,
		D2D1::PixelFormat(
		DXGI_FORMAT_B8G8R8A8_UNORM,
		D2D1_ALPHA_MODE_PREMULTIPLIED
		)
		);

    ComPtr<ID2D1Bitmap1> otherBmp;
    m_d2d1DeviceContext->CreateBitmap(
		size, 
		nullptr,
		0,
		&prop1,
		&otherBmp
		);

    D2D1_RECT_U rect = {0, 0, 3264, 2448};
    D2D1_POINT_2U point = {0, 0};
    hr = otherBmp->CopyFromBitmap(&point, m_d2d1TargetBitmap.Get(), &rect);  
	
    D2D1_MAP_OPTIONS options = D2D1_MAP_OPTIONS_READ;
    D2D1_MAPPED_RECT mappedRect;
    hr = otherBmp->Map(options, &mappedRect);
    memcpy(pixels, mappedRect.bits, W * H * 4);
    otherBmp->Unmap();

Answer 8

Thanks. I resolve my issue using WicBitmap and then create RenderTarget from Wic.

Answer 9

Pragos answered worked for me, but I figured I'd post this for anyone else stumbling across it. The pitch value of the mapped rect isn't guaranteed to be the same as the width * 4. I mean, it might always work out to be the same, but the existence of the pitch member to D2D_MAPPED_RECT implies that it isn't. So this code should be safe, in all cases:

for (int y = 0; y < H; y++)

{

    // assuming pixels is a char*

    memcpy(pixels + (y * W * 4), mappedRect.bits + mappedRect.pitch, W * 4);

}