GMFBridge, two graphs, and Video Mixing Renderer 9

Question

It seems like this question is asked a lot and, for whatever reason, I cannot find a solution that works.  I apologize in advance if my search skills missed the appropriate answer, but I am asking anyway.

I am creating capture and preview graphs bridged with the GMFBridgeController filter.  By default, I try to use VMR9 in the preview graph and fall back on the GDI renderer if I cannot.  If I can use VMR9, I use it in windowless mode.

For simplicity, I am going to leave audio out.

The capture graph looks like:

[Video Source] -->

[Smart Tee] -->

[ASF Writer] [GMF Bridge Sink]

The preview graph looks like:

[GMF Bridge Source] -->

[Smart Tee] -->

[VMR9] [Sample Grabber] --> [Null Renderer]

As I said, the VMR9 is windowless and drawing to a window.  The WM_PAINT handler for the window only calls BeginPaint(), passes the HWND and HDC to IVMRWindowlessControl9::RepaintVideo(), and calls EndPaint().  The WM_DISPLAYCHANGE handler calls IVMRWindowlessControl9::DisplayModeChanged().

Everything runs fine until I lock the workstation.  I know, I know, VMR9 loses it surface, etc. etc.  However, SOMETIMES the preview graph recovers and continues running.  Other times, the preview graph just stops and RepaintView() just draws a black rectangle.  The capture graph continues to run unaffected.

I never receive WM_DISPLAYCHANGE in my window.

I tried cancelling default handling for EC_DISPLAY_CHANGED, and the thread I have running to monitor events from the two graphs does receive EC_DISPLAY_CHANGED.  I tried a variety of things in the monitor thread.

For example, I call IGMFBridgeController::BridgeGraphs(0, 0), call IMediaControl::Stop(), disconnect the VMR9 input pin and then reconnect it using IGraphBuilder::Connect(), then call IMediaControl::Run(), and IGMFBridgeController::BridgeGraphs(sink, feeder).

I also tried simply stopping and starting the graph.  Etc. etc. etc.  Nothing helps.

Without handling EC_DISPLAY_CHANGED the graph may recover on its own once or twice, but will, invariably, finally stop and no longer run.

The only thing I have not tried (yet) is completely tearing down the graph and rebuilding it in response to EC_DISPLAY_CHANGED.  Nothing I have seen indicates that I have to do that, but no solutions I have seen actually work, so...

Answer 1

For posterity, it looks like I to do the following:

* Cancel default processing of EC_DISPLAY_CHANGED in the preview graph when building it.
* Start a thread to monitor the preview graph for events.
* Fire off a message to my window when the thread receives EC_DISPLAY_CHANGED and let the thread shut itself down.
* Unbridge the graphs.
* Stop and completely destroy the preview graph.
* Rebuild the preview graph from scratch.
* Start the monitor thread again.
* Start the preview graph.
* Rebridge the graphs.

Funny that, when using a single graph, VMR9 was perfectly capable of shutting down the graph, restarting it, and trashing the video file.  But, suddenly I have to hold its hand and do everything manually when it is isolated in a second graph where it can't damage the file.

Answer 2

Well, I am back to square one.  I was originally testing with a Logitech web cam, then switched to an Axis PTZ camera and I am back to the original issue.

I have placed a lot of trace information in my code, and I see the following sequence...

>>> Media event (2): 00000016
>>> Preview graph received display change notification.
>>> Monitor thread shutting down.
>>> Destroying preview graph.
>>> Rebuilding preview graph.
>>> Monitor thread running.
>>> Media event (2): 00000053
>>> Media event (2): 0000000D
>>> Media event (2): 0000000E
>>> Graph is running.
>>> Forcing video repaint.

So, my monitoring thread receives the display change notification when I lock the workstation and shuts itself down after posting a Windows message to the VMR9 "clipping" window (which is just the app's main window) telling it to rebuild the preview graph.

The window stops the graph, completely destroys it, then rebuilds it and restarts the monitoring thread, and starts the graph.  The monitor thread immediately receives Render Device Set, Clock Changed, and Paused.

I then explicitly check the state of the graph and only output "Graph is running" if IMediaControl::GetState() returns S_OK and sets the OAFilterState parameter to State_Running.

The last line indicates that the clipping Window received WM_PAINT and that it is calling IVMRWindowlessControl9::RepaintVideo().

After all of that, however, I just get a black rectangle.  The capture graph is still running and writing the video file.  But, I am just unable to restart the video preview.

I am at a complete loss here, and any help would be appreciated greatly.

Answer 3

Bump.  Still hoping for an answer.

I created a TransInPlace filter that just counts the samples that pass through it and outputs a message every 50 samples.  Refer to my original graph diagram, the debug filter is between the [GMF Bridge Source] and the [Smart Tee] in my preview graph.

I also added code to check the state of the bridge sink, the bridge source, and the VMR9.

My output now reads...

>>> (Event Debug) Samples processed: 200
>>> (Event Debug) Samples processed: 250
>>> (Event Debug) Samples processed: 300
>>> (Event Debug) Samples processed: 350
>>> Forcing video repaint.
>>> Media event (2): 00000016
>>> Preview graph received display change notification.
>>> Monitor thread shutting down.
>>> Destroying preview graph.
>>> Rebuilding preview graph.
>>> Monitor thread running.
>>> Media event (2): 00000053
>>> Graph is running.
>>> Filter is running.
>>> Filter is running.
>>> Filter is running.
>>> Forcing video repaint.
>>> Media event (2): 0000000D
>>> Media event (2): 0000000E

So, essentially, the debug filter sees samples come across.  The preview graph receives EC_DISPLAY_CHANGE and the application tears down the graph, rebuilds the graph, and restarts the graph.  The preview graph reports it is running, the GMF sink reports it runing, the GMF source reports it is running, and the VMR9 reports it is running.

Yet, there it is, the debug filter does not see any samples come across after the rebuild.

Again, this is after locking the workstation once or twice.  So, the code is properly bridging the graphs after rebuilding the preview graph.  The GMF bridge just seems to stop delivering samples to the preview graph.

For added fun, I noticed that every time I rebuild the preview graph, the application leaks 10-20 MB of memory.  So, that's fun.

Answer 4

Bump again.  This is ****ing infuriating.

I created debug filters to monitor the streams and output a message every 50 samples received.  I ultimately figured out that the [Smart Tee] in the capture graph would stop delivering samples on its preview pin.  One sentence in the Smart Tee documentation I missed.

OK, fine, so I replaced the [Smart Tee] filters with [Infinite Tee] filters.  That's great.  Now, my Logitech camera works perfectly reliably.  The preview graph receives EC_DISPLAY_CHANGED when locking the workstation and moving the window across monitors, the window rebuilds the graph, and the preview continues playing.  Awesome.

Then, I try an Axis camera and the preview graph no longer receives EC_DISPLAY_CHANGED, the window is obviously no longer able to rebuild the graph, and the VMR9 just freezes even though the preview graph and all of its filters report that they are running, and the debug filters show samples being delivered to the VMR9.

What determines how EC_DISPLAY_CHANGED gets delivered to the application?  Why would it matter whether I am using Smart Tee's or Infinite Tee's?  And, that is literally the only difference.

Answer 5

SlakCoder wrote:

>

>Why would it matter whether I am using Smart Tee's or Infinite Tee's? 

>And, that is literally the only difference.

 

The Smart Tee, despite its rather appealing sounding name, has a very

specific purpose.  It provides split capture and preview pins for sources

that only have a capture pin.  The sole purpose of the preview pin is to

strip the timestamps from the samples so that frame are rendered as fast as

possible, without waiting for the timestamp.  In a preview situation,

that's generally what you want.

 

Except for that very specific situation, the Smart Tee is a dumb tee.  For

general purpose splitting of a stream into multiple destinations, the

Infinite Tee is the correct choice.

--

Tim Roberts, [e-mail]

Providenza & Boekelheide, Inc.

 

---

Tim Roberts, VC++ MVP Providenza & Boekelheide, Inc.

Answer 6

Tim,

Thanks for responding, and I understand all of that.  To summarize my previous posts and restate the problem...

I receive EC_DISPLAY_CHANGED for all cameras with which I test when I use Smart Tees in the graph.  However, the Smart Tee will stop delivering samples on the preview pin after a random number of preview graph rebuilds.  Destroying and rebuilding the preview graph is the ONLY way I have discovered to recover from a surface loss in the VMR9.  Nothing short of that will reliably recover the preview functionality.

So, the answer that became obvious after sifting through the documentation again was to switch to the Infinite Tee filter.  However, when I switch to the Infinite Tee filter, the graph stops receiving EC_DISPLAY_CHANGED when I test with an Axis PTZ camera.  When testing with a Logitech web cam and Sony Ipela PTZ camera, the graph does receive EC_DISPLAY_CHANGED.

The question you quoted was really asking: why would the type of tee determine whether the graph receives EC_DISPLAY_CHANGED?

What is absolutely INSANELY bizarre about the whole situation, though, is that the Axis source filter has nothing to do with the preview graph on which I am listening for EC_DISPLAY_CHANGED.  The preview graph is isolated from the capture graph that contains the Axis source filter by the GMF Bridge.  Yet, for some reason that I have yet to uncover, the preview graph will not receive EC_DISPLAY_CHANGED if the capture graph contains the Axis source filter and both graphs are using Infinite Tees.

I have gone over the graphs again and again.  All of the filters are exactly the same.  The same Axis source filter, the same MPEG decoder filters, everything.  The only thing that changes is the type of tee, and, suddenly, the preview graph stops receiving EC_DISPLAY_CHANGED.

Answer 7

Bump.

Answer 8

Bump.

Answer 9

SlakCoder wrote:

>

>Bump.

 

I doubt that anyone knows.  You're talking about very specific combinations

of components causing differences that the typical DirectShow programmer

wouldn't encounter.  I'm afraid you're just in for some long debugging

sessions, tracing through code that doesn't belong to you.

--

Tim Roberts, [e-mail]

Providenza & Boekelheide, Inc.

 

---

Tim Roberts, VC++ MVP Providenza & Boekelheide, Inc.

Answer 10

SlackCoder wrote :

"I have gone over the graphs again and again.  All of the filters are exactly the same.  The same Axis source filter, the same MPEG decoder filters, everything.  The only thing that changes is the type of tee, and, suddenly, the preview graph stops receiving EC_DISPLAY_CHANGED."

To understand the odd behaviours you are seeing, you have to understand two points :

1) The Infinite pin tee has also a specific behaviour described by Gerraint Davis in one of the threads nearby. It does not actually duplicate the samples, but passes them in series to the various branches of the tee. While less "odd" than the smart tee "pseudo samples", it still means that what you do on one branch may impact the second one...

2) The GMF Bridge does also avoid copying samples, but does pass them to the downstream graph.

To avoid this total coupling, you must introduce a filter which does copy of samples : color space converter or any transform filter (but not transinplace) downstream of the infinite pin tee.

I am not sure it will help solving your problem :)

 

---

Michel Roujansky, http://www.roujansky.com

Answer 11

Thanks for the responses.  I ultimately had a coworker go over the code yesterday because, clearly, I was missing something and needed a fresh perspective.  And, ultimately, all of my work was pointless.

We have been using the GDI renderer instead of VMR9 for quite a while because we had never really been able to solve the problem with the VMR9 losing its DirectX resources.  When we had to start supporting Axis PTZ cameras, we found that, with the GDI renderer, either we just received a black window or the application would crash.

I experimented with VMR9 again and it seemed to work fine with the Axis driver except that we were back to VMR9 restarting the graph and trashing the video file.  Hence this project to isolate the preview graph with GMF Bridge and attempt to solve the associated problems.

As it turns out, the problem was that I was relying solely on DirectShow to handle graph construction.  The Axis driver was adding a MJPEG Decoder DMO to the graph after adding the Axis Source.  Relying solely on DirectShow resulted in the DMO only being added to one branch of the Tee.

The GDI renderer actually works fine with the Axis Source.  The problem was the DMO needed to be connected before the Tee.  So, we had to write code specific to that situation to check the graph for the presence of that DMO...which, thankfully, is named "Axis MJPEG Decoder DMO"...and connect it between the Axis Source and the Tee.

The final graph ended up looking like:

[Axis Source] --->

[Axis MJPEG Decoder] --->

[Inf Tee] --->

[GDI Renderer] [ASF Writer] [Sample Grabber] ---> [Null Renderer]

The solution does not work with the GMF Bridge, however, so we're still rolling with the GDI Renderer.  My test application hooks up the ASF Writer first, so the DMO was ending up on that branch rather than the branch that connects to the GMF Bridge Sink.  If I modify the test application to connect the DMO before the Tee in the capture graph, RenderStream fails when trying to render the Tee in the preview graph to the VMR9.  Whatever.