Windows >
Software Development for Windows Client Forums
>
DirectShow Development
>
Video and audio drifting out of sync
Video and audio drifting out of sync
- I have a TV playing/recording application that has been working fine up to recently. I now have a problem with the audio gradually getting in front of the picture and it appears that the longer a station is played the larger the gap gets. After an hour it is up to 3-4 seconds.
A standard stream buffer engine source graph is used for playing the stations so it looks like
SBE source --> ffdshow video decoder --> vmr9
--> ffdshow audio decoder --> directsound renderer
It happens on all stations on different frequencies and on different machines. If I stop the station playing (ie tear down the graph) and start it up again the audio and picture are back in sync, at least to start with.
Looking at the audio stats from the directsound renderer there is a large amount of silence inserted ( 13 seconds over an hour) and a number of breaks. Is this significant? I've only recently displayed the audio stats in the application so these figures may have always been like this.
It must be something I have changed but the code for this graph has not changed for some considerable time so I could do with some tips as to where to start looking.
Regards
Steve
All Replies
- IMO, you must first identify if the problem is created when recording or when playing back.
For me, silence insertion and audio getitng ahead of video, this looks like you are missing some audio samples in the recording.
On your SBE source, are you reading from a random start, or or you always trying to read the latest recording?
Michel Roujansky, http://www.roujansky.com - It looks like it's just on the SBE source side.
The application I have allows you to start recording while watching a program so I played a station until it was well out of sync and then started recording. The recording played back perfectly even though the same broadcast live was out of sync.
The recording feature uses an SBE sink recorder object.
On the SBE source side I always start reading from the latest recording.
Regards
Steve - The problem may be inaccurate time stamps being applied by the ffdshow decoders. You could investigate the time stamp values before and after the decoder and try a different decoder.
www.chrisnet.net Some questions about looking at the timestamps.
Is the best way to put a sample grabber each side of a decoder?
Should the sample timestamps be the same each side of a decoder?
Should the timestamps each side of the audio decoder be the same as each side of the video decoder.
Sorry if the questions are stupid but I don't have much experience of debugging graphs at this level even though I have some fairly complicated graphs. Normally I load the filters, connect the relvant pins and hey presto everything works.
I don't think the decoders can be at fault as I have been using the same ones for months without any problem. Also they are playing recorded TV and DVD's ok.
Any help is much appreciated.
Steve- Yes, the easiest method to debug this is to put a SampleGrabber each side of the decoder.The timestamps coming out of the decoder should be the same unless the stream itself contains internal timestamps that may be more accurate. This is possibly the case for MPEG-2 and other MPEG formats. I'm not fully sure what the SBE strips out and what it leaves.It sounds as though the main problem may be that the playback rate overall is slightly too fast when watching the live content, leading to buffer underruns in the audio renderer.
www.chrisnet.net - I'll put in the sample grabbers and see what that produces.
If the problem is with the playback rate how can I tinker around with it?
Steve - Check the stats on the audio renderer. See if it is operating with a Live source clock.
www.chrisnet.net - The audio renderer stats say it is running with the graphs reference clock (0x04).
- Unfortunately it looks like there is no option of having a live source clock with the StreamBufferSource filter. It would seem like the best option may be to playback the "live" content slightly less live, try seeking to a position that is 5 or 10 seconds prior to the end using IStreamBufferMediaSeeking.
www.chrisnet.net - I've discovered that the playback of live TV is geting further and further behind real time.
I ran the same channel for 12 hours (ie the same graph) and after that time the audio was about 5 seconds in front of the video but the whole thing was about 10 minutes behind what was actually being broadcast on the channel at the time.
So I think that both the audio and video are 'slipping' but at slightly different rates.
If I use IMediaSeeking to move the playback forward the audio/video gets back in sync again and you can also end up back at real time so I'm pretty sure the sink graph is fine.
It's as if the SBE source graph can't keep up with the SBE sink graph but doesn't drop samples so it simply gets further and further behind real time.
This application has a number of threads running as well as the media playback. Is it possible I have unwittingly introduced a thread of processing that means this particular graph (ie the SBE source graph) is being starved of resources? It must be something peculiar to this graph as DVD playback doesn't suffer from the same problems. The background threads run during DVD playback as well.
This was all working fine (for months) up to 3-4 weeks ago.
Appreciate your input Chris.
Regards
Steve - Because it had been working previously and you state that the code itself hasn't changed, then I would start looking for other things on the system that may have changed. Look for new software installations, Windows updates, etc. Because both the audio and the video are falling behind real-time I don't think it is a CPU loading issue as they both run on their own threads. You can confirm individual thread CPU usage by using a tool such as Process Explorer, if it gets near 100/ncores then you have a bottle neck.Setting up a clean test system may also be a worthwhile test at this point.
www.chrisnet.net - On the other hand, it's playing back a live stream using a local clock. Perhaps the two clocks were very close in frequency and now they are not.
G
- But typically clock frequencies don't change.
www.chrisnet.net I seem to have fixed the problem.
After doing some reading up I decided to try setting the Graph Manager clock to the DirectSound filter and now the audio and video are perfectly in sync.
What I don't quite understand is why the Graph Manger was not doing this anyway as, after a live souce (which an SBE source graph doesn't appear to have) it should choose the DirectSound filter.
As this was working for a long time I wonder if the Graph Manager did originally choose the DirectSound filter for the clock but at some point it stopped doing that.
The audio stats now report that there is no rate matching (0x00).
Any ideas?
Steve
