VMR9: Renders only 50% of the time in Fullscreen and freezes when using alt+F4

Question

System I'm using : Windows XP, sp2. Visual Studio 2005 professional. Windows SDK 6.0. VMR9 in windowless mode

Requirements : I need a video renderer which will run on a game window both in fullscreen and windowed. I don't need any mixing, only one input stream and no bitmap overlays, but I've chosen VMR9 as the rest of the game engine uses D3D9. I need the video rendering to be cross-windows, so not just in XP. I also need something that can render compressed videos. I'd prefer to avoid writing an allocator-presenter for this.

the issue : Running in a window works perfectly fine. I create the filter graph and do all the set up right when I play the video and release the graph at the end of it. If I flip into fullscreen (true fullscreen, not a stretched window) before running the video, about 50% of the time, the video will not render. When I press alt+enter (which the engine flips from fullscreen to windowed mode), I see the video image in fullscreen then when the game flips back to windowed, it's paused and it never hits any of my paused statements.

I've thrown in several logging statements in the WM_PAINT message handling and find that the HRESULT from the render returns S_OK every time. WM_PAINT is being called often enough that it should be painting just fine, as it does when not in fullscreen.

I've tried with both compressed and uncompressed AVI files as well as WMV files, all with the same issue. They all run fine in a window, but fail 50% of the time in fullscreen and freeze when flipping between fullscreen and window.

Filter graph : I'm using the generic filter graph set up the default for AVI files. CoCreateInstance(CLSID_FilterGraph, NULL, CLSCTX_INPROC, IID_IGraphBuilder, (void **)(&m_pGraphBuilder)); I'm primarily using WMV but I've also used the Cinepak Codec with AVIs. I haven't gotten any other decompression filters to work properly either.

Playing the video : this is when I set up all my graphs, the VMR, and the rectangle. I pulled the code off MSDN directly, adding asserts, logging statements, and changing variable names:
playVideo( const wchar_t *videoname)
{   
    //ensure no other video's playing right now
    //also releases the graph and resources
    if ( m_bIsActive )
        stopVideo();

    //build a filter graph for the video to play with
    buildFilterGraph();

    HWND hwnd = GET_APP->getWindow()->getWindowHandle();
    HRESULT hr = S_OK;
    assert (SUCCEEDED(hr));

    assert ( m_pGraphBuilder );
    assert( m_pMediaControl );

    hr = InitWindowlessVMR(hwnd, m_pGraphBuilder, &m_pWindowlessControl);
    assert( hr == S_OK );
   
    calculateDestinationRect();

    // create the filter graph
    hr = m_pGraphBuilder->RenderFile(videoname, NULL);
    assert( hr == S_OK );

    // get the duration
    m_pMediaSeeking->GetDuration(&m_fDuration);


    // play the file
    m_pMediaControl->Run();
}

Building the graph
buildFilterGraph()
{
     bool fullscreen = GET_DISPLAY->isFullScreen();

    HWND hwnd = GET_APP->getWindow()->getWindowHandle();
    HRESULT hr;

    CoInitializeEx(NULL, COINIT_MULTITHREADED);

    // Get an instance of the graph builder
    CoCreateInstance(CLSID_FilterGraph, NULL, CLSCTX_INPROC, IID_IGraphBuilder,
        (void **)(&m_pGraphBuilder));

    assert ( m_pGraphBuilder );

    // Get the references to interfaces
    hr = m_pGraphBuilder->QueryInterface(IID_IMediaControl, (void **)(&m_pMediaControl));
    assert( m_pMediaControl );

    hr = m_pGraphBuilder->QueryInterface(IID_IMediaSeeking, (void **)(&m_pMediaSeeking));
    assert( m_pMediaSeeking );

    m_pGraphBuilder->QueryInterface(IID_IMediaEventEx, (void **)&m_pMediaEvent);
    assert ( m_pMediaEvent );
    m_pMediaEvent->SetNotifyWindow((OAHWND)hwnd, WM_GRAPHNOTIFY, 0);
}

Initializing the windowless VMR :
HRESULT
InitWindowlessVMR(
                                    HWND hwndApp,                  // Window to hold the video.
                                    IGraphBuilder* pGraph         // Pointer to the Filter Graph Manager.
                                    )
{
    if (!pGraph)
    {
        return E_POINTER;
    }

    IVMRWindowlessControl* pWc = NULL;
    IBaseFilter*   m_pVmr = NULL;
    // Create the VMR.
    HRESULT hr = CoCreateInstance(CLSID_VideoMixingRenderer9, NULL,
        CLSCTX_INPROC_SERVER, IID_IBaseFilter, (void**)&m_pVmr);
    assert ( SUCCEEDED(hr) );

    // Add the VMR to the filter graph.
    hr = pGraph->AddFilter(m_pVmr, L"Video Mixing Renderer");
    assert ( SUCCEEDED(hr) );

    // Set the rendering mode. 
    IVMRFilterConfig* pConfig;
    hr = m_pVmr->QueryInterface(IID_IVMRFilterConfig9, (void**) &pConfig);
    assert ( SUCCEEDED(hr) );

    hr = pConfig->SetRenderingMode(VMR9Mode_Windowless);
    pConfig->Release();        
    assert ( SUCCEEDED(hr) );

    // Set the window.
    hr = m_pVmr->QueryInterface(IID_IVMRWindowlessControl9, (void**)&m_pWindowlessControl);
    assert ( SUCCEEDED(hr)) ;

    m_pWindowlessControl->SetVideoClippingWindow(hwndApp);

    return hr;
}

Destination rectangle :
void
calculateDestinationRect()
{
    // Find the native video size.
    long lWidth, lHeight;
    HWND hwnd = GET_APP->getWindow()->getWindowHandle();
    HRESULT hr = S_OK;
    hr = m_pWindowlessControl->GetNativeVideoSize(&lWidth, &lHeight, NULL, NULL);

    assert (SUCCEEDED(hr));
    {
        RECT rcSrc;
        // Set the source rectangle.
        SetRect(&rcSrc, 0, 0, lWidth, lHeight);
       
        // Get the window client area.
        GetClientRect(hwnd, &m_rcDestination);
        // Set the destination rectangle.
        SetRect(&m_rcDestination, 0, 0, m_rcDestination.right, m_rcDestination.bottom);
       
        // Set the video position.
        hr = m_pWindowlessControl->SetVideoPosition(&rcSrc, &m_rcDestination);
    }
}

In the video manager's message handler : Before paint is called, the engine invalidate's the window's surface rect.
case WM_PAINT:
        {
            PAINTSTRUCT ps;
            HDC         hdc;
            RECT        rcClient;
            GetClientRect(hWnd, &rcClient);
            hdc = BeginPaint(hWnd, &ps);
            if (isActive())
            {
                // Find the region where the application can paint by subtracting
                // the video destination rectangle from the client area.
                // (Assume that g_rcDest was calculated previously.)
                DEBUG_LOG(LOG_MESSAGE, L"repainting video: client area");
                HRGN rgnClient = CreateRectRgnIndirect(&rcClient);
                HRGN rgnVideo  = CreateRectRgnIndirect(&GET_VIDEOMGR->getDestinationRect()); 
                CombineRgn(rgnClient, rgnClient, rgnVideo, RGN_DIFF); 

                // Paint on window.
                HBRUSH hbr = GetSysColorBrush(COLOR_BTNFACE);
                FillRgn(hdc, rgnClient, hbr);

                // Clean up.
                DeleteObject(hbr);
                DeleteObject(rgnClient);
                DeleteObject(rgnVideo);
               
                assert(m_pWindowlessControl);
                DEBUG_LOG(LOG_MESSAGE, L"repainting video: VMR");
                HRESULT hr = m_pWindowlessControl->RepaintVideo(hWnd, hdc);
                assert( hr == S_OK );
                DEBUG_LOG(LOG_MESSAGE, L"repainting video: HR: %d", hr);

                //check to see if the video has ended
                LONGLONG currTime = 0;
                hr = m_pMediaSeeking->GetCurrentPosition(&currTime);
                assert( hr == S_OK );

                if ( currTime >= m_fDuration )
                {
                    stopVideo();
                }
            }
            EndPaint(hWnd, &ps);
        }
        break;

Log file during render:
=|= MESSAGE WARNING 5 =|= repainting video: invalidate rect
=|= MESSAGE WARNING 5 =|= repainting video: client area
=|= MESSAGE WARNING 5 =|= repainting video: VMR
=|= MESSAGE WARNING 5 =|= repainting video: HR: 0

Result :
all asserts pass. All HRESULTS come back S_OK. The rectangle is being invalidate. I've run the video test a good 50 times while tinkering with the code. The result's still the same no matter which video codec I use, either Cinepak, uncompressed, or WMV. Are there any solutions to get the game to render video in fullscreen short of writing my own allocator-presenter? I've tried the samples provided with the SDK and searched multiple forums without an answer.

Answer 1

If it's of any consolation, I too experiance this, but haven't found a solution... yet!

Answer 2

The usual advice for this is to use a borderless, topmost window that occupies the full screen.  That is what I do.

---

Please use Vote As Helpful (green up arrow at top-left of posts) and Mark As Answer where appropriate.
My dshow site is http://tmhare.mvps.org.

Answer 3

My first attempt was using the IVideoWindow and flipping it between fullscreen and windowed mode. However that caused the video to flicker and the game to ocassionally lock up and I'd have to use my task manager just to shut it down. I also read that the IVideoWindow isn't compatible across versions of windows. Same with the VMR7. I've had this issue with both VMR7 and 9. The sound plays, just no image.

I can see I can't change dymanically between windowed and windowless, and I can disable alt+enter when the video is playing, but what about if the user hits alt+tab? alt+tab freezes the video, just like alt+enter does. I am also using true fullscreen, and I don't believe I can draw a window on top of that (unless there's some trick I don't know about?)

Answer 4

Alright, so I finally got it to work. In the end I did have to use a renderless VMR to make this work, building my own allocator/presenter (which really wasn't as hard to do as I would have thought.

I used the vmr9allocator sample in the sdk (you'll probably have to link the windows sdk into the project file like I did) as an example and extracted the code I needed. As I have a renderer in my engine already, I simply retrieved both the IDirect3D9* and the IDirect3DDevice9* from my engine's renderer and basically took over it during video playback. Then once I was done, I released the two interfaces back to the renderer and let the renderer reset the device back to its original settings.

now as I didn't use the SmartPointers the example used, I had to add a Release for each interface created (except the IDirect3D9 and the IDirect3DDevice9 interfaces) as well as any calls using GetBackBuffer(...) and GetSurfaceLevel(...). There's only one of each but they're easy to find. They allocate their own interfaces and you have to release them before resetting the D3D device else D3D throws a fit that the pool still has allocations.

Only problem with the code is alt+enter stops the video. I haven't figured that one out yet, so we're just disabling that functionality during playback. alt+tab isn't allowed during playback so that solves that problem too

hope this helps anyone who has the same problem I did.