Causes of delay in video stream

Question

I've come across a slightly tricky problem that has left my team and myself scratching our heads.

We have a fairly simple graph that runs absolutely fine on almost every machine we've tried it with - it hasn't been without problems but the current graph and code for assembling the graph is reliable and robust.

However on one pair of computers the video runs with 2-3 second delay. I've taken the graph out of the application into a test shell and run it independently and it works just fine with maybe 0.2-0.3 second delay. I've put in code to simulate the rest of the application's background activity and it has made no difference at all.

CPU usage sits at about 50% while the graph is running, this is a single core AMD chip running at 2.9MHz with plenty of memory and Windows XP so I can't see anything that should be causing this problem. What I do know is that it *is* something in the application, the graph used to run fine until a recent upgrade but the graph component was not changed. Going back to the old build is unfortunately not an option. What I can't figure out is what the key change is as none of the changes should be causing this.

So getting to the main question - what are the general causes of delay in the video stream?

I believe I've eliminated any possible errors in the graph itself as the cause. The CPU is not overloaded in any way. The graph itself appears otherwise responsive.

The graph itself looks like this:

Video Source (in this case an Intensity Pro card) -> AVI Decompressor -> Custom Filter (basic DSP) -> Smart Tee (preview) -> Sample Grabber -> Video Renderer

The custom filter takes up about 10% CPU on these machines but I've tried the graph with and without to no effect on the delay. The graph is intended to be switchable to include recording off the capture pin of the smart tee.

Answer 1

The graph itself looks like this:

Video Source (in this case an Intensity Pro card) -> AVI Decompressor -> Custom Filter (basic DSP) -> Smart Tee (preview) -> Sample Grabber -> Video Renderer

The custom filter takes up about 10% CPU on these machines but I've tried the graph with and without to no effect on the delay. The graph is intended to be switchable to include recording off the capture pin of the smart tee.

Thought I replied to this already, but don't see my post (apologies if it shows up twice). Are you saying you see the delay with this graph too?

Video Source (in this case an Intensity Pro card) -> AVI Decompressor -> Smart Tee (preview) -> Sample Grabber -> Video Renderer

That, of course, would rule out your filter. I have never used the Sample Grabber (as it is deprecated, so I wrote my own), but I have used the IVMRWindowlessControl9::GetCurrentImage method. If you call that too often, it can create just the type of delay you are describing, apparently because it reads from VRAM, and that can be surprisingly slow. Are you grabbing frequently? Maybe changing the interval between grabs will give you some useful info.

---

Stevens Miller http://www.withoutsupervision.com

Answer 2

Thanks for the reply,

The graph - when tested outside of our application - works just fine, either in a test harness or through graphedit. The sample grabber is used very occasionally, in a graph lifecycle maybe just 2 or 3 times. We use it to achieve a freeze-frame and still capture.

The headache is what external influence could cause the delay as I don't believe it is the graph itself. Given the lack of CPU load I am struggling to understand what could be causing the issue anyway although I do appreciate that Windows XP is rather good at hiding background activity so just taking the CPU load from the task manager is potentially misleading.

Answer 3

The task manager reports overall load, so if it says that's low, it's probably not some hidden XP activity stealing your cycles. I suppose you could run another program at the same time, one you are sure has no issues, and see if it also slows down. I am guessing it is more likely to be something about how frames are produced on your system. Are the drivers for your Intensity Pro the same across all machines involved? Does your application do any device initialization that is not present in your test set-ups? Are you doing any multi-threaded work that could leave a critical resource held while waiting for something else?

Referring back to your original question ("what are the general causes of delay in the video stream?"), the causes I have experienced have included: excessive reading from VRAM; filters that take longer per sample than the interval between samples from a push source producing samples on its own thread; one or another of the many ways I have found to screw up synchronized threads of my own creation.

Beyond that, all I can offer is general debugging advice, which it sounds to me like you don't much need (that is, I get the impression you know how to debug a program as well as the next guy does). FWIW, though, I'd consider using the "Moose in Alaska," method (that name comes from an article I read years ago in "Communications of the ACM"), which is basically Newton's Method applied to code: start lopping off half your code at a time (replacing it with stubs that will at least let it run) and see which half creates the problem. Lop that half into halves (that is, into quarters) and repeat until you find the line that creates the problem. Yes, it's rather like turning a bull loose in a china shop (to mix my animal metaphors, but blame Grace Hopper for the fact that we call them "bugs" in the first place), but it often gets the job done.

---

Stevens Miller http://www.withoutsupervision.com

Answer 4

Thanks again,

As it happens the test harness took the opposite approach to your Moose - putting in the bare bones and then adding in large chunks of functionality but strangely the fault never appeared in the harness.

Ironically it appears we have solved the issue but the dangerous aspect to it is that I'm not 100% sure of the how we solved it.

I found a timer on the imaging form of the application that was enabled and triggering every 50ms, the event code was brief but entirely unnecessary for the sites suffering the problem. I included emulation of this in a test harness and proved that the event was harmless and did not affect the video stream but as it had been a bit of messy programming by one of my team it was isolated and disabled as inappropriate.

The video stream renderer output was also shrunk down from full screen to a window - again tested in the harness and proven a marginal success (it reduced the delay by about a third but only from about 0.6s down to 0.2s and that is with real certified seat-of-the-pants-finger-in-the-air timing).

When built and tested on the customer site the delay had mysteriously disappeared. Even when the render was set back to full screen the delay was minimal.

There are only two possible solutions I can think of - the timer was in fact the problem in the first place, or the compilation of the original, problematic build was faulty in some way.

Given more time I'd be running more tests but as this is a live customer site with no direct access and the fault is not reproducible elsewhere, it looks like this remains something of a mystery.

Answer 5

Ironically it appears we have solved the issue but the dangerous aspect to it is that I'm not 100% sure of the how we solved it.

Brother, I feel your pain.

Given more time I'd be running more tests but as this is a live customer site with no direct access and the fault is not reproducible elsewhere, it looks like this remains something of a mystery.

There's a time for being a perfectionist, and there's a time to just be glad it works. If the customer is happy, then I'd say you done good. Thanks for the up-vote on my answer!

---

Stevens Miller http://www.withoutsupervision.com