locked
MediaPlayer problem playing back http video stream

    Question

  • I am working on moving my Windows Phone application to Metro.  The key feature of the app is streaming video.  I have the video working correctly using the MediaPlayer when the source contains audio.  For example, setting the source of the media player to  http://nzp-wb02.si.edu/pandacam1 works great.  If I set it to the source of a camera that does not have audio such as  http://nzp-wb02.SI.EDU/7WB6W6I4WZSG the stream starts and then gets into this weird buffering state.  It shows a capture from the first frame and the keeps trying to load.  This works correctly using the mediaelement on Windows Phone.  MediaElement and MediaPlayer in Metro both exhibit the same behavior.  Thank you.

    Scott


    Scott Lock, MVP C# Principal Consultant - Excella Consulting President, Capital Area .Net Users Group (Caparea.net)

    Monday, March 12, 2012 1:00 PM

Answers

  • Hello Scott,

    Based on my initial observation I believe that the problem is due to the way that the source media server is transmitting the time stamps of the video feed. In my test application (code below) I can see the "Position" reported by the ME roll over to zero. At this time the video freezes but the play state continues to be maintained. From my testing WMP detects a similar problem with the time stamps and stops playback all together.

    Believe it or not this behavior is by design and is a common problem with continuously available media streams. Internally we calculate the current time stamp using 100 ns units and store the result in a 64 bit signed integer. This is enough precision for about 30 days, give or take, of continuous playback. Once the time stamp register rolls over the behavior becomes "undefined". That said the behavior of the Metro style app MediaElement appears to be consistent with behavior I have seen before.

    In this scenario the ME receives an initial time stamp of 1,160,000 from the source stream. This time stamp increases monotonically based on the time stamp specified by the data stream. The data stream reaches a value of about 1,200,000 and then (for an unknown reason) resets the time stamp to zero. Internally the ME has scheduled the next frame to be shown on the screen at 1,200,001. Since the time stamp of the current frame is zero, 1, 2, 3, etc. the ME starts to drop frames until the time stamp hits 1,200,001. This never happens because the source stream resets the time stamp before this position can be hit.

    To me this is a problem with the source stream. The ASF specification clearly states that a time stamp must progress monotonically forward, cannot go backward and cannot reset. If for some reason the time stamp must be changed, a discontinuity flag must be set in the sample with the new timestamp range. When we receive the discontinuity we automatically reset the current time stamp value to match the new value of the stream at the location of the discontinuity.

    As per the fragmented nature of the currently media code base we have different APIs that react differently to this all too common situation. We have tried different "band aids" in the past to handle these bad streams. However, this is not necessarily the right thing to do. The real solution here is to educate the owner of the source stream on the problems that they are creating by not following the ASF specification.

    <Page
        x:Class="MediaElementPositionContinues.BlankPage"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:local="using:MediaElementPositionContinues"
        xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
        xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
        mc:Ignorable="d">
    
        <Grid Background="{StaticResource ApplicationPageBackgroundBrush}">
            <MediaElement x:Name="_ME" HorizontalAlignment="Left" Height="430" Margin="66,47,0,0" VerticalAlignment="Top" Width="765" PosterSource="Assets/SplashScreen.png" Source="http://nzp-wb02.si.edu/pandacam1" IsLooping="True" AutoPlay="True" CurrentStateChanged="_ME_CurrentStateChanged" MediaFailed="_ME_MediaFailed"/>
            <TextBox x:Name="_TB" HorizontalAlignment="Left" Height="18" Margin="66,507,0,0" TextWrapping="Wrap" Text="0" VerticalAlignment="Top" Width="141"/>
            <TextBox x:Name="_tbState" HorizontalAlignment="Left" Height="18" Margin="242,507,0,0" TextWrapping="Wrap" Text="" VerticalAlignment="Top" Width="141"/>
        </Grid>
    </Page>

            private void _ME_CurrentStateChanged(object sender, RoutedEventArgs e)
            {
                _tbState.Text = _ME.CurrentState.ToString();
    
                _DT.Interval = new TimeSpan(0, 0, 1);
    
                _DT.Tick += _DT_Tick;
    
                _DT.Start();
            }
    
            void _DT_Tick(object sender, object e)
            {
                _TB.Text = _ME.Position.TotalSeconds.ToString();
            }
    
            void Suspend()
            {
                _tbState.Text = "Suspend";
            }
    
            private void _ME_MediaFailed(object sender, ExceptionRoutedEventArgs e)
            {
                _tbState.Text = e.ErrorMessage;
            }

    I hope this helps.

    James


    Windows Media SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/





    Tuesday, April 17, 2012 9:42 PM

All replies

  • Hi Scott,

    Can you give me a bit more information on the type of stream you are using and format? Is the only difference one has Audio and the other doesn't? I am going to try to dig a bit deeper into what is happening with the second stream.  Does it always stay in the buffering state and never get back into playing? I'll look at it from our side but getting some details on the stream would help as well.

    Thanks,

    MArco


    Marco

    Monday, March 12, 2012 5:27 PM
  • Marco,

    I wish I could give you more info on the feeds.  They are not managed by me.  They are property of the National Zoo.  I built my Windows Phone app - http://www.windowsphone.com/en-US/apps/bacc5dc6-711e-49f5-b51a-ce24ee39dd91 using the mediaelement control.  All the cameras work flawlessly.  The links I had represent the two types of feeds that I can see any difference.  Over 18 feeds have no sound.  They appear to be streaming from Windows Media Server, but I cannot tell what version. 

    I've been experimenting with the XAML and the C# behind the controls.  Here's the XAML that I tried.  Try dropping this into a sample page:

    <MediaPlayer AutoPlay="True" Source="http://nzp-wb02.SI.EDU/7WB6W6I4WZSG" BorderBrush="Goldenrod" x:Name="meCam" Grid.ColumnSpan="2" HorizontalAlignment="Left" Height="480" Margin="0,0,20,0" VerticalAlignment="Top" Width="640"/>

    If you replace the source with http://nzp-wb02.si.edu/pandacam1 you will see what I mean.

    Scott


    Scott Lock, MVP C# Principal Consultant - Excella Consulting President, Capital Area .Net Users Group (Caparea.net)

    Tuesday, March 13, 2012 10:50 AM
  • Marco,

    Any further thoughts on this? 

    Scott


    Scott Lock, MVP C# Principal Consultant - Excella Consulting President, Capital Area .Net Users Group (Caparea.net)

    Wednesday, March 14, 2012 12:29 AM
  • Sorry for the delay Scott. We are actively looking at this and hope to know more soon.

    Marco

    Friday, March 16, 2012 6:02 PM
  • Marco,

    Any movement?  Is there something I can do on my side to troubleshoot?

    Scott


    Scott Lock, MVP C# Principal Consultant - Excella Consulting President, Capital Area .Net Users Group (Caparea.net)

    Tuesday, March 20, 2012 2:10 AM
  • Marco,

    Any word?

    Scott


    Scott Lock, MVP C# Principal Consultant - Excella Consulting President, Capital Area .Net Users Group (Caparea.net)

    Friday, March 23, 2012 4:15 AM
  • Sorry for the delay Scott. The second stream actually does play after waiting for quite some time. In our internally test it would vary but sometimes as long as 15 minutes, which is odd. Have you encountered other streams this happens on? I'm trying to find a pattern or something special about these URL's that would cause this.

    Marco

    Wednesday, April 11, 2012 7:46 PM
  • I've only been working with URLs from Fonz.org so I do not have something to compare them to.  That being said, they all run immediately after a short buffer time (milliseconds) using the mediaelement control and built in media player on Windows Phone 7.  Also, if you open the URLs directly they open and run correctly in Media Player on both Windows 7 and Windows 8.

    Unfortuately, my application for Windows 8 is severly crippled if I can't get the cameras to run.  What are your next steps to diagnose this?


    Scott Lock, MVP C# Principal Consultant - Excella Consulting President, Capital Area .Net Users Group (Caparea.net)

    Thursday, April 12, 2012 12:51 PM
  • Hello Scott,

    Are you still having problems with this issue? If so please let me know and I will do what I can to help.

    Thanks,

    James


    Windows Media SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

    Friday, April 13, 2012 12:57 AM
  • James,

    I am still having this issue.  I really have no insight into the issue other than what Marco has said.  The stream works well on Windows Phone using MediaElement and it works fine on Windows 8 in the Media Player itself.  But using the MediaElement control or the MediaPlayer control in XAML in a Metro app, it doesn't work.  There are 2 feeds that do work.  The difference is that they have sound.  I do not have access to the servers (owned and operated by Smithsonian) so I can't tell you what version of Windows Server they are streaming from and if there is a difference.  If you are part of Marco's team or if you can get his testing results, I would suggest starting there.  I'll do anything I can from this side, but it's tough.  It's a black box to me.

    Thank you,

    Scott


    Scott Lock, MVP C# Principal Consultant - Excella Consulting President, Capital Area .Net Users Group (Caparea.net)

    Friday, April 13, 2012 1:07 PM
  • Hello Scott,

    Based on my initial observation I believe that the problem is due to the way that the source media server is transmitting the time stamps of the video feed. In my test application (code below) I can see the "Position" reported by the ME roll over to zero. At this time the video freezes but the play state continues to be maintained. From my testing WMP detects a similar problem with the time stamps and stops playback all together.

    Believe it or not this behavior is by design and is a common problem with continuously available media streams. Internally we calculate the current time stamp using 100 ns units and store the result in a 64 bit signed integer. This is enough precision for about 30 days, give or take, of continuous playback. Once the time stamp register rolls over the behavior becomes "undefined". That said the behavior of the Metro style app MediaElement appears to be consistent with behavior I have seen before.

    In this scenario the ME receives an initial time stamp of 1,160,000 from the source stream. This time stamp increases monotonically based on the time stamp specified by the data stream. The data stream reaches a value of about 1,200,000 and then (for an unknown reason) resets the time stamp to zero. Internally the ME has scheduled the next frame to be shown on the screen at 1,200,001. Since the time stamp of the current frame is zero, 1, 2, 3, etc. the ME starts to drop frames until the time stamp hits 1,200,001. This never happens because the source stream resets the time stamp before this position can be hit.

    To me this is a problem with the source stream. The ASF specification clearly states that a time stamp must progress monotonically forward, cannot go backward and cannot reset. If for some reason the time stamp must be changed, a discontinuity flag must be set in the sample with the new timestamp range. When we receive the discontinuity we automatically reset the current time stamp value to match the new value of the stream at the location of the discontinuity.

    As per the fragmented nature of the currently media code base we have different APIs that react differently to this all too common situation. We have tried different "band aids" in the past to handle these bad streams. However, this is not necessarily the right thing to do. The real solution here is to educate the owner of the source stream on the problems that they are creating by not following the ASF specification.

    <Page
        x:Class="MediaElementPositionContinues.BlankPage"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:local="using:MediaElementPositionContinues"
        xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
        xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
        mc:Ignorable="d">
    
        <Grid Background="{StaticResource ApplicationPageBackgroundBrush}">
            <MediaElement x:Name="_ME" HorizontalAlignment="Left" Height="430" Margin="66,47,0,0" VerticalAlignment="Top" Width="765" PosterSource="Assets/SplashScreen.png" Source="http://nzp-wb02.si.edu/pandacam1" IsLooping="True" AutoPlay="True" CurrentStateChanged="_ME_CurrentStateChanged" MediaFailed="_ME_MediaFailed"/>
            <TextBox x:Name="_TB" HorizontalAlignment="Left" Height="18" Margin="66,507,0,0" TextWrapping="Wrap" Text="0" VerticalAlignment="Top" Width="141"/>
            <TextBox x:Name="_tbState" HorizontalAlignment="Left" Height="18" Margin="242,507,0,0" TextWrapping="Wrap" Text="" VerticalAlignment="Top" Width="141"/>
        </Grid>
    </Page>

            private void _ME_CurrentStateChanged(object sender, RoutedEventArgs e)
            {
                _tbState.Text = _ME.CurrentState.ToString();
    
                _DT.Interval = new TimeSpan(0, 0, 1);
    
                _DT.Tick += _DT_Tick;
    
                _DT.Start();
            }
    
            void _DT_Tick(object sender, object e)
            {
                _TB.Text = _ME.Position.TotalSeconds.ToString();
            }
    
            void Suspend()
            {
                _tbState.Text = "Suspend";
            }
    
            private void _ME_MediaFailed(object sender, ExceptionRoutedEventArgs e)
            {
                _tbState.Text = e.ErrorMessage;
            }

    I hope this helps.

    James


    Windows Media SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/





    Tuesday, April 17, 2012 9:42 PM
  • James,

    Thank you for the detailed response and your time on this, but here is what I don't get.  If what you are saying is true then would the stream work on IE with an embeded media player (see www.fonz.org), in the mediaelement control for Windows Phone 7 (see National Zoo app in marketplace), or in WMP on Windows 8 desktop?  If what you are saying is true then I would expcect this to fail in all places.  How do you explain that?

    Scott


    Scott Lock, MVP C# Principal Consultant - Excella Consulting President, Capital Area .Net Users Group (Caparea.net)

    Wednesday, April 18, 2012 5:03 PM
  • Hello Scott,

    "As per the fragmented nature of the current media code base we have different APIs that react differently to this all too common situation. We have tried different "band aids" in the past to handle these bad streams. However, this is not necessarily the right thing to do."

    In other words, WMP, WP7 and Metro all use a slightly different codebase. WMP is extremely (and IMHO way too) tolerate of mal formed data. I don't know anything of the WP7 codebase so I can't comment on this. I would guess that SL has yet a different behavior.

    -James


    Windows Media SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

    Thursday, April 19, 2012 11:59 PM
  • James,

    I worked on this last night to see if I could replicate what you were saying.  I think that I did, although I have to tell you that the end result sucks.  I understand the purist view that WMP and WP7 Silverlight controls are too loose, but they WORK!  I think that the team should consider doing whatever the WMP and Silverlight teams have done to handle less then perfect streams.  It really stinks that you have something that works across every other platform you make, coming from a product (Windows Media Server) that you produced and not have it work for Windows 8 Metro. 

    The only thing I can do on my side is get clever and try to roll my own .asf stream reader.  This is really disappointing.

    Scott


    Scott Lock, MVP C# Principal Consultant - Excella Consulting President, Capital Area .Net Users Group (Caparea.net)

    Friday, April 20, 2012 1:46 PM
  • Hello Scott,

    Thanks for your feedback. I certainly understand how having differing behavior between the various media platforms can be frustrating.  The network source team is aware of this difference in behavior. I honestly can't tell you if they will decide to handle this scenario differently or not.

    Keep in mind that you don't need to write your own ASF stream parser to work around this issue. You simply need to write a custom MFT that will rebase the timestamps of the already parsed stream. If you are experienced at writing DirectShow filters or MFTs this should be very easy for you to do in just a few hours time.

    I hope this helps,

    James


    Windows Media SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

    Tuesday, April 24, 2012 12:38 AM
  • James,

    Thanks for at least making the team aware.  I appreciate that.  I'm not experienced at all in writing a custom MFT or DirectShow filters.  Can you point me in the right direction?

    Scott


    Scott Lock, MVP C# Principal Consultant - Excella Consulting President, Capital Area .Net Users Group (Caparea.net)

    Tuesday, April 24, 2012 12:45 AM
  • Hello Scott,

    Take a look at the Grey Scale MFT as part of this sample:

    http://code.msdn.microsoft.com/windowsapps/Media-Capture-Sample-adf87622

    -James


    Windows Media SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

    Friday, April 27, 2012 12:41 AM