locked
Using MediaStreamSource to play streaming raw H264 frames RRS feed

  • Question

  • Hi all:

    Win8.1 add new class MediaStreamSource that can be used to play audio and video data http://msdn.microsoft.com/library/windows/apps/dn282716

    But when I use it to play the streaming raw H264 frames, sometimes it's perfect to play for several hours, but sometimes the render frames are broken or hanging in just several seconds. The SampleRequested events still occur, but the frame can't be displayed on media element correctly.

    We develop similar app for Windows Phone 8 and using Silverlight MediaStreamSource. No such issue exists so we don't think it's our frame issue.

    Does anyone try to use the MediaStreamSource and meet the same issue? How do we check the issue if it's about the media pipeline?




    • Edited by Yanger317 Thursday, September 5, 2013 2:17 PM
    Thursday, September 5, 2013 2:16 PM

Answers

  • Hello,

    Without being able to reproduce the problem and taking a look at your code, I can only wager a guess. Make sure that you are reporting discontinuities correctly. If a discontinuity occurs and the codec is not made aware video playback can get corrupt. Another possibility is related but different. Make sure that the video stream always starts with a time stamp of zero. Make sure that all time stamp information increases monotonically. Any change in the time stamp time base (i.e. going from 1000000 back to 0) must be accompanied with a discontinuity. Also keep in mind that the time stamp is represented by a signed 64 bit integer. This integer will eventually roll over even if you start at zero (about 30 days). Once the timestamp value rolls over the behavior of the media stack is completely undefined. Because of this we always recommend that you restart your timestamp base (with the proper discontinuity) every 15 to 28 days to avoid the possibility of an integer rollover.

    I hope this helps,

    James


    Windows SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

    Friday, September 6, 2013 10:50 PM
    Moderator
  • Thanks James.

    I think we have found the root cause that is about timestamp error. The correct timestamp should be the interval between the time of MediaStreamSource.Starting and submitting the frame in MediaStreamSource.SampleRequested (in our real time application). We used timestamp of server side coded frame and it may fall behind  time of submitting the frame, for the network latency.


    Saturday, September 7, 2013 3:39 PM

All replies

  • I'll ask our media guy to look at this.

    Matt Small - Microsoft Escalation Engineer - Forum Moderator
    If my reply answers your question, please mark this post as answered.

    NOTE: If I ask for code, please provide something that I can drop directly into a project and run (including XAML), or an actual application project. I'm trying to help a lot of people, so I don't have time to figure out weird snippets with undefined objects and unknown namespaces.

    Friday, September 6, 2013 1:26 PM
    Moderator
  • Hello,

    Without being able to reproduce the problem and taking a look at your code, I can only wager a guess. Make sure that you are reporting discontinuities correctly. If a discontinuity occurs and the codec is not made aware video playback can get corrupt. Another possibility is related but different. Make sure that the video stream always starts with a time stamp of zero. Make sure that all time stamp information increases monotonically. Any change in the time stamp time base (i.e. going from 1000000 back to 0) must be accompanied with a discontinuity. Also keep in mind that the time stamp is represented by a signed 64 bit integer. This integer will eventually roll over even if you start at zero (about 30 days). Once the timestamp value rolls over the behavior of the media stack is completely undefined. Because of this we always recommend that you restart your timestamp base (with the proper discontinuity) every 15 to 28 days to avoid the possibility of an integer rollover.

    I hope this helps,

    James


    Windows SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

    Friday, September 6, 2013 10:50 PM
    Moderator
  • Thanks James.

    I think we have found the root cause that is about timestamp error. The correct timestamp should be the interval between the time of MediaStreamSource.Starting and submitting the frame in MediaStreamSource.SampleRequested (in our real time application). We used timestamp of server side coded frame and it may fall behind  time of submitting the frame, for the network latency.


    Saturday, September 7, 2013 3:39 PM
  • is represented by a signed 64 bit integer. This integer will eventually roll over even if you start at zero (about 30 days). Once the timestamp value rolls over the behavior of the media stack is completely

    2^64 * 100ns is a bit more than 30 days.  (More like 58 thousand years--an order of magnitude greater than recorded human history.)  Even cutting that in half to the 2^63 positive integers, leaves plenty of margin. 

    Real world streams may have much shorter periods, and since one may be joining an existing stream at any point in its period (e.g., two minutes before it wraps), one does need to deal with it. For example, the PTS in an MPEG-2 transport stream is 33 bits wide and rolls over every 26 hours or so (IIRC).  I extended it to 64 bits--under the assumption that sequential samples are never too far apart--like so:  RegisterExtender.cs.

    BTW, a big thank you to whoever came up with the new MediaStreamSource for WinRT.  The old Silverlight/Windows Phone version can be a real pain to deal with.

    Saturday, October 12, 2013 5:41 PM