locked
[UWP][C#] What does MediaStreamSourceSampleRenderedEventArgs.SampleLag mean exactly? RRS feed

  • Question

  • The document says it means "The lag with which a sample was rendered.".

    If a sample has a time stamp of 10,000 ms (as TimeSpan), but is rendered at 20,000 ms, I thought SampleLag would be 10,000 ms.  This does not appear to be the case.

    For example, when I put an app in the background, the rendering appears to stop almost immediately.  When I bring the app to the foreground 10 seconds later, and the rendering starts immediately.  I thought I would see SampleLag to be about 10 seconds, but it is 0.5 seconds or so. 

    I am asking this because I would like use SampleLag the assess the video latency of real-time video streaming.


    Hong

    Wednesday, June 6, 2018 7:28 PM

All replies

  • Hi Hong,

    The SampleLag per my understanding is a different thing. The sampling is the conversion between analog and digital, and the lag is related to sample bandwidth(or frequency). The example you mentioned seems to be the delay when you play a music file, not sampling. So you might not on the right direction here.

    Hmm, not so sure if it helps, if you find video latency, have you enabled the following https://docs.microsoft.com/en-us/windows/uwp/design/controls-and-patterns/media-playback#enable-low-latency-playback ?

    Best regards,

    Barry


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Monday, June 25, 2018 9:02 AM
  • Thank you for trying to help, Barry.

    The samples are provided by the code that knows the exact sampling frequency. In other words, the code has 100% control of the samples.

    mp.MediaPlayer.RealTimePlayback does not seem to make any difference as confirmed by others.


    Hong

    Monday, June 25, 2018 11:01 AM
  • Hello Hong,

    When you suspend the app the video stops playing. This is basically like pushing the "pause" button. When the app moves to the foreground it resumes and this is like pressing the play button. Because of this the SampleLag value isn't incremented because the video wasn't actually playing during this time. In other words, the player has it's own clock and when you suspend the app the clock stops and when you resume the app the clock starts where it left off. There is no lag because to the app no time has actually passed when the app was suspended.

    I hope this helps,

    James 


    Windows SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

    Wednesday, June 27, 2018 12:35 AM
  • Thank you for the insight, James.

    What does SampleLag mean exactly?  In other words, how is it calculated?  What is the reference of this lag?


    Hong

    Wednesday, June 27, 2018 1:39 AM
  • Hi Hong,

    The SampleLag is used to allow you to determine if samples are arriving late from the network. If the network latency (lag) increases the samples may stat arriving late and the SampleLag parameter will be incremented. You can use this parameter to determine if you need to choose a lower or higher bitrate due to the network latency.

    I hope this helps,

    James


    Windows SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

    Thursday, June 28, 2018 1:00 AM
  • Hi James,

    The samples are provided by app code.  MediaPlayer has not idea where they come from.  They can come from a local file, a network stream source, random data generated by code for testing...   Therefore, how can MediaPlayer assumes the stream is from the network?  Where does it get the network information? The app may run on a machine has not network connection, but the app can still provide the sample data whenever it is requested. 


    Hong

    Friday, June 29, 2018 11:16 PM
  • Your question is about:

    MediaStreamSourceSampleRenderedEventArgs.SampleLag

    Correct?

    Unless I'm missing something this event argument class has nothing to do with the MediaPlayer. It is specific to the MediaStreamSource. The MediaStreamSource was originally implemented to allow for various custom network streaming scenarios that were not inherent in the platform at the time. So while the MediaStream source could theoretically source data from a file it is typically used in a networking scenario. In a local file playback scenario the lag will likely never be large enough to increment the SampleLag value. Because of this discussing the SampleLag parameter in the context of local file playback with the MediaStreamSource is moot. 

    -James


    Windows SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

    Tuesday, July 3, 2018 1:21 AM
  • Hi James,

    Yes, you are right it is related to MediaStreamSource.

    I am mentioning MediaPlayer because the instance of MediaStreamSource is used as the source of a MediaPlayer:
          myMediaPlayerElement.Source = MediaSource.CreateFromMediaStreamSource(myMediaStreamSource).
    Therefore, if I understand correctly, it is the MediaPlayer (wrapped by a MediaPlayerElement) that does all the work including requesting samples, rendering them. and provide SampleLag information.

    I mentioned a local file for the sole purpose of making my point that the MediaPlayer or MediaStreamSource has no clue about network conditions.  I prefer not disclosing how the app gets samples fed to MediaStreamSource, hence MediaPlayer to avoid confusing people because it is irrelevant to the discussion. No matter how the app gets samples, there is no way for MediaStreamSource or MediaPlayer to have any clue about the source type (network, local file, random data...).


    Hong

    Tuesday, July 3, 2018 1:45 AM
  • Hi Hong,

    I think we are way off topic her but I'll try again to explain. I agree that the MeidaPlayer doesn't know where the samples are coming from. However, it knows everything it needs to calculate the lag. It knows the time stamp of the sample, when it was processed by the MediaSource and when it was rendered. Using these three pieces of information it can calculate an estimate for the sample latency (lag).

    As per my previous post "In a local file playback scenario the lag will likely never be large enough to increment the SampleLag value. Because of this discussing the SampleLag parameter in the context of local file playback with the MediaStreamSource is moot."

    In other words, the only time you are going to see a high lag or latency is when the sample's original timestamp and actual render time are significantly out of sync. The only time that this is likely to happen is when the sample's timestamp originates outside of the MediaSource such as a network streaming server.

    Again, as per my previous post, "The MediaStreamSource was originally implemented to allow for various custom network streaming scenarios that were not inherent in the platform at the time." This means that some of the settings / events / etc. may not apply to all scenarios. It will likely not appy in the local file playback or sample generation scenario. 

    I hope that I've cleared up the ambiguity but I understand that this is a bit of a complex subject. If you feel that you need more specific information I would be happy to discuss it further in the context of a direct engagement. Feel free to open a ticket with my team by going to http://aka.ms/storesupport, clicking on "contact us" and then selecting "UWP app development". Note: There may be a charge for opening a ticket. If you have an MSDN subscription you may be eligible for free support from Microsoft. 

    Thanks,

    James


    Windows SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

    Thursday, July 5, 2018 8:58 PM
  • Thanks for the effort, James.

    As you described, MediaStreamSource reports MediaStreamSample with the following information:

    1. Buffer (i.e. video frame data).
    2. Time stamp (TimeSpan).
    3. Duration (TimeSpan).

    The way you described about SampleLag is what I originally thought and interpreted, but found it did not explain my observation.  That is the reason for me to post this question.  I originally wanted the app to make some decisions based on that number, but found it was not what I thought.  It is less than 1 second while the video latency is greater than 10 seconds based the time stamp.

    Per your explanation, if the time stamp is 5 seconds, and the frame is rendered at 10 seconds, SampleLag would be 5 seconds, but it was not per my observation.  I will test it again and report it back. 


    Hong

    Friday, July 6, 2018 3:06 AM