none
Dynamic Packaging to HLS V3

    Question

  • I noticed Nick mentioned that HLS V3 is now available.  Is this available for Dynamic Packaging now?  If so, what is the new manifect accessor (ie. "(format=m3u8-appl)")?

    Thank you,

    Justin

    Tuesday, April 1, 2014 11:13 PM

Answers

  • Hi Justin,

    Yes, I can confirm that the HLS v3 enabled server has been deployed into production.  However, some of my early adopters did find a bug for source content in which the smooth manifest uses repeat tags in the audio stream.  That is, content that has an audio chunk list which looks like:

    <c d=20000000 r=42 />

    Will have playback failures.  Not all encoders produce audio with r tags, so not all content will be affected.  When the fix for this is put into production, there will be no user action required, any new HLS v3 streams will be corrected. In the event you are using a CDN, you may want to use a new Locator to break the CDN cache such that you are getting the new, fixed, content from the Origin server, and not the buggy older responses from the CDN.  This is why the official doc has not been released/updated, I want to avoid having to explain it in support ticket after support ticket. J

    That said, here is a draft of my blog, which contains all the essentials.  You’re welcome to use the feature if your content is not problematic:

    Azure Media Services now supports a down-level version of HLS, HLS v3.  As of version 4, September 2011, Apple/HLS added support for multiple-languages:  Alternative Audio Renditions these were called.  Multi-language support was the motivation behind choosing HLS v4 two years ago for our service.  At that time, Android was seeing a lot of active development and with its largely international customer base, it was assumed that Android would adopt HLS v4 for these multi-lingual markets.  Not so, as history would prove.  Here we are two and a half years later and HLS v4 support is only available on the very latest Android drops.  So, instead, we are reaching back and dusting off the HLS v3 spec, November 2010, and building backward compatibility.  We are also motivated by the connected TV space, which has a lower adoption rate for device updates.  

    To use HLS v3, simply use:

    *.ism/manifest(format=m3u8-aapl-v3)

    This will mux the default audio language into the video.

    But what about multi-language scenarios, since HLS v3 doesn’t support those?

    HLS v4 allows video and audio to be delivered in a decoupled fashion (like Smooth, DASH and HDS), whereas HLS v3 must have the audio muxed into the video data, and both are delivered as a single stream.  Since HLS v3 does not have native multi-language support, but our server does, we were able to allow content-owners to set the audio track that they want muxed in the URL using the ‘audioTrack’ attribute.  If you don’t set anything, we just mux in the default audio.

    You can force the language that you want muxed into the video by using the audioTrack attribute.

    It looks like this:

    *.ism/manifest(format=m3u8-aapl-v3,audioTrack=audio_eng)

    *.ism/manifest(format=m3u8-aapl-v3,audioTrack=audio_spa)

    Or

    *.ism/manifest(format=m3u8-aapl-v3,audioTrack=audio)

    *.ism/manifest(format=m3u8-aapl-v3,audioTrack=audio_1)

    Depending on if your audio languages were detected/defined in the source media.

    You can get the audioTrack name from the smooth manifest, or HLS v4 manifest.  You can set your own track names with the title=”audio_spa” attribute in the .ism file used for your multi-bitrate MP4 assets. .  Similarly, you can use systemLanguage=”spa” to set the Language attribute in the manifests.  Add the attributes to the line in the .ism which has the src=”yourAudio.mp4”.  You can also do some renaming in the .ism/.ismc of Smooth assets if you’re feeling brave.  Sorry there’s no support for this type of manipulation in the platform, perhaps some of you would like to contribute to the Azure Media Services .NET SDK Extensions project on GitHub?

    So now HLS works on all Android devices, right?

    Nope, not a chance.  Android is highly fractured and has varying levels of implementation for streaming media.  The media stacks are different from device to device, and even within a single device the media stack can be duplicated/different.  If you’ve been trying to make this work for a while, you’ve seen that the same Android version may work differently on different handsets; that’s because each manufacturer has the option of tweaking the OS.  All to say, it’s not an easy problem to solve, and as a stream provider, the best I can do is make sure my streams are conformant – device support/upgrades is up to Android.

    We also did some deeper testing of 4.1.2 debug builds which demonstrated that the quality of the underlying native implementation was low for that release.  Some manufacturers had forked the code and done some fixes, but generally, I would not expect devices 4.1.2 and below to work.  This is consistent with the findings of other streaming server and player framework providers.

    For the Android devices with version 4.2.2, it would seem that there are varying degrees of implementation.  Some adaptive bitrate logic implementation is particularly poor (dropped frames are not triggering bitrate switching, latching to a particular bitrate or bitrate switching causes a re-start).  Other devices with 4.2.2 do reasonable well, and the latest Android drops are much better. 

    The MSDN doc will have a table on the device testing we were able to accomplish as part of building this feature, it is in no way extensive, but does cover a good number of popular devices that can be purchased today.  Just to be clear, the goal of HLS v3 is to provide spec-compliant HLS streams which pass Apple validator and other Transport Stream compliance suites; not to ‘work on all Android devices’, so your millage may vary, take it up with the other guys.  J

    On the connected TV side, we’ve gotten good feedback from our partners thus far.

    Regards,

    Nick Drouin

    Program Manager, Azure Media Services


    Wednesday, April 2, 2014 5:45 PM
    Owner
  • Got an answer back fast.

    Eric at JWPlayer said that you just need to add the following to your embed code.

    type:"HLS"

    Their hosted JWPlayer publishing tool at jwplayer.com is not automatically doing that for our URL format. I put in a request for them to support that. For now you can use the tool to configure the player, grab the embed code and script tag and put it into your web page and then update the type:"HLS" property to make it work. 

    The reason it is not working is because we use a RESTful URL for our manifest files, and we don't put any extension in the file URL.  It messes up some players that parse the URL for .msu8 extensions.

    Tuesday, April 22, 2014 9:52 PM
    Owner
  • Yeah, that is because the flash player doesn't work in a local file.  You can host it locally in an ASP.NET web server, though.

    This is what they say in the JW Player docs:

    This means the page with JW Player embedded is not loaded from a webserver.  Due to various online-offline security restrictions, such a setup is not supported in the Flash rendering mode.  Either upload your page to a webserver or run a local webserver to circumvent this."

    For the heck of it, I threw a link up to your HLS V3 version on our website in a test page.  Playing great!

    http://mail.yogadownload.com/test-trek-hls.html

    Justin

    • Proposed as answer by reklats Tuesday, April 22, 2014 9:54 PM
    • Marked as answer by JG - YDL Tuesday, April 22, 2014 10:03 PM
    Tuesday, April 22, 2014 9:26 PM

All replies

  • I found the answer and dynamic packaging is supported.  It isn't formally documented that I have found yet, but Nick posted a twitter post with the new accessor, it is (format=m3u8-aapl-v3).

    Working great by the way. :)

    JWPlayer is also working well with the HLS v3 (it now plays the audio).

    Thank you Media Services team for getting this out there!

    Justin

    Wednesday, April 2, 2014 12:29 AM
  • Hi Justin,

    Yes, I can confirm that the HLS v3 enabled server has been deployed into production.  However, some of my early adopters did find a bug for source content in which the smooth manifest uses repeat tags in the audio stream.  That is, content that has an audio chunk list which looks like:

    <c d=20000000 r=42 />

    Will have playback failures.  Not all encoders produce audio with r tags, so not all content will be affected.  When the fix for this is put into production, there will be no user action required, any new HLS v3 streams will be corrected. In the event you are using a CDN, you may want to use a new Locator to break the CDN cache such that you are getting the new, fixed, content from the Origin server, and not the buggy older responses from the CDN.  This is why the official doc has not been released/updated, I want to avoid having to explain it in support ticket after support ticket. J

    That said, here is a draft of my blog, which contains all the essentials.  You’re welcome to use the feature if your content is not problematic:

    Azure Media Services now supports a down-level version of HLS, HLS v3.  As of version 4, September 2011, Apple/HLS added support for multiple-languages:  Alternative Audio Renditions these were called.  Multi-language support was the motivation behind choosing HLS v4 two years ago for our service.  At that time, Android was seeing a lot of active development and with its largely international customer base, it was assumed that Android would adopt HLS v4 for these multi-lingual markets.  Not so, as history would prove.  Here we are two and a half years later and HLS v4 support is only available on the very latest Android drops.  So, instead, we are reaching back and dusting off the HLS v3 spec, November 2010, and building backward compatibility.  We are also motivated by the connected TV space, which has a lower adoption rate for device updates.  

    To use HLS v3, simply use:

    *.ism/manifest(format=m3u8-aapl-v3)

    This will mux the default audio language into the video.

    But what about multi-language scenarios, since HLS v3 doesn’t support those?

    HLS v4 allows video and audio to be delivered in a decoupled fashion (like Smooth, DASH and HDS), whereas HLS v3 must have the audio muxed into the video data, and both are delivered as a single stream.  Since HLS v3 does not have native multi-language support, but our server does, we were able to allow content-owners to set the audio track that they want muxed in the URL using the ‘audioTrack’ attribute.  If you don’t set anything, we just mux in the default audio.

    You can force the language that you want muxed into the video by using the audioTrack attribute.

    It looks like this:

    *.ism/manifest(format=m3u8-aapl-v3,audioTrack=audio_eng)

    *.ism/manifest(format=m3u8-aapl-v3,audioTrack=audio_spa)

    Or

    *.ism/manifest(format=m3u8-aapl-v3,audioTrack=audio)

    *.ism/manifest(format=m3u8-aapl-v3,audioTrack=audio_1)

    Depending on if your audio languages were detected/defined in the source media.

    You can get the audioTrack name from the smooth manifest, or HLS v4 manifest.  You can set your own track names with the title=”audio_spa” attribute in the .ism file used for your multi-bitrate MP4 assets. .  Similarly, you can use systemLanguage=”spa” to set the Language attribute in the manifests.  Add the attributes to the line in the .ism which has the src=”yourAudio.mp4”.  You can also do some renaming in the .ism/.ismc of Smooth assets if you’re feeling brave.  Sorry there’s no support for this type of manipulation in the platform, perhaps some of you would like to contribute to the Azure Media Services .NET SDK Extensions project on GitHub?

    So now HLS works on all Android devices, right?

    Nope, not a chance.  Android is highly fractured and has varying levels of implementation for streaming media.  The media stacks are different from device to device, and even within a single device the media stack can be duplicated/different.  If you’ve been trying to make this work for a while, you’ve seen that the same Android version may work differently on different handsets; that’s because each manufacturer has the option of tweaking the OS.  All to say, it’s not an easy problem to solve, and as a stream provider, the best I can do is make sure my streams are conformant – device support/upgrades is up to Android.

    We also did some deeper testing of 4.1.2 debug builds which demonstrated that the quality of the underlying native implementation was low for that release.  Some manufacturers had forked the code and done some fixes, but generally, I would not expect devices 4.1.2 and below to work.  This is consistent with the findings of other streaming server and player framework providers.

    For the Android devices with version 4.2.2, it would seem that there are varying degrees of implementation.  Some adaptive bitrate logic implementation is particularly poor (dropped frames are not triggering bitrate switching, latching to a particular bitrate or bitrate switching causes a re-start).  Other devices with 4.2.2 do reasonable well, and the latest Android drops are much better. 

    The MSDN doc will have a table on the device testing we were able to accomplish as part of building this feature, it is in no way extensive, but does cover a good number of popular devices that can be purchased today.  Just to be clear, the goal of HLS v3 is to provide spec-compliant HLS streams which pass Apple validator and other Transport Stream compliance suites; not to ‘work on all Android devices’, so your millage may vary, take it up with the other guys.  J

    On the connected TV side, we’ve gotten good feedback from our partners thus far.

    Regards,

    Nick Drouin

    Program Manager, Azure Media Services


    Wednesday, April 2, 2014 5:45 PM
    Owner
  • I added the -v3 to the manifest URL but all I get is an error that states "The server cannot service the request because the media type is unsupported.".
    Tuesday, April 22, 2014 3:42 PM
  • I might be able to help you if you provide more detail.  Have you been able to stream with any of the other manifest types (smooth streaming or HLS 4 (basically not including the -v3)?  Do you have a smooth streaming (ISM) file in the asset and URL?  Are you using multiple bit rates and what did you encode with?

    Justin

    Tuesday, April 22, 2014 3:58 PM
  • This is our first foray into Media Services. We are trying to get streaming working via JWPlayer. 

    Encoded via the Portal using the 'iOS and PC/Mac' preset for the portal. This resulted in an "intermediate" asset (my understanding is that this is smooth streaming) and the "final" asset which is the HLS. 

    Here is the URL for the final HLS asset:

    http://trekmediasvcdevuseast01.origin.mediaservices.windows.net/c7a284a1-b135-4cc7-b250-25bc9b3818a9/domane_tech_web_en-m3u8-aapl.ism/Manifest(format=m3u8-aapl)

    If I paste that into a browser, it will download the manifest file. We have been unable to get it to work in JWPlayer at all (it sounded like it should play video, but no audio until we get HLS v3 working).

    If I click play in Media Services portal, all I get is a black window. I had assumed that the built-in player that pops up didn't support it, but maybe this is indicating a larger failure?

    Here is the URL for the intermediate asset:

    http://trekmediasvcdevuseast01.origin.mediaservices.windows.net/3b019279-85ca-4cc2-ae17-ad475edd1c47/domane_tech_web_en.ism/Manifest

    If I click play in the Media Services portal, the video plays.

    Tuesday, April 22, 2014 4:07 PM
  • This is what was created in the asset's container in our storage account. 

    domane_tech_web_en-m3u8-aapl-1096000.ismx
    domane_tech_web_en-m3u8-aapl-1096000.m3u8
    domane_tech_web_en-m3u8-aapl-1096000.ts
    domane_tech_web_en-m3u8-aapl-1596000.ismx
    domane_tech_web_en-m3u8-aapl-1596000.m3u8
    domane_tech_web_en-m3u8-aapl-1596000.ts
    domane_tech_web_en-m3u8-aapl-2346000.ismx
    domane_tech_web_en-m3u8-aapl-2346000.m3u8
    domane_tech_web_en-m3u8-aapl-2346000.ts
    domane_tech_web_en-m3u8-aapl-3496000.ismx
    domane_tech_web_en-m3u8-aapl-3496000.m3u8
    domane_tech_web_en-m3u8-aapl-3496000.ts
    domane_tech_web_en-m3u8-aapl-496000.ismx
    domane_tech_web_en-m3u8-aapl-496000.m3u8
    domane_tech_web_en-m3u8-aapl-496000.ts
    domane_tech_web_en-m3u8-aapl-746000.ismx
    domane_tech_web_en-m3u8-aapl-746000.m3u8
    domane_tech_web_en-m3u8-aapl-746000.ts
    domane_tech_web_en-m3u8-aapl-96000.ismx
    domane_tech_web_en-m3u8-aapl-96000.m3u8
    domane_tech_web_en-m3u8-aapl-96000.ts
    domane_tech_web_en-m3u8-aapl.ism
    domane_tech_web_en-m3u8-aapl.m3u8
    index.htm

    Tuesday, April 22, 2014 4:30 PM
  • So your issue probably that you have packaged ahead of time and not used dynamic packaging.  With Dynamic Packaging you can deliver Smooth Streaming, HLS 3, HLS 4, and DASH (which will be awesome when the client end catches up) all from the same smoothing stream source and this is done on demand.

    I test your intermediate asset is not in a position for dynamic packaging either.  The asset needs to be smooth streaming asset, but composed of MP4 files for each bit rate (if you are using multiple bit rates, which you should to get the gains of adaptive streaming).  Specifically these are H.264 MP4 adaptive bitrate sets, which is hard to achieve with some encoders (need to match group of pages between files), but is easy with the right preset from Media Services.

    The typical one people are using is: H264 Adaptive Bitrate MP4 Set 720p

    I ended up building my own custom encoding preset off of this so that I could vary not just the bit rate, but also the resolution.  (People are more familiar with 720p, 480p, then bit rate numbers and I think I had a reason this worked better with JW Player)   If you end up wanting to do a custom preset, feel free to reach out again as they are a bit tricky.

    Then once you have this asset encoded, point to the ISM with the manifest URL and using the v3 ending.  JW Players flash player will then be able to stream it.  If flash is not available, the HLS 3 streaming is not supported.  For our solution (which just went live today!) we have a Silverlight player using Smooth Streaming, JW Player using Flash with HLS 3, HLS4 for iOS, and for no flash or Silverlight we use JW Player as an HTML5 player and add the different bit rate files to a list to allow the user to manually select.  This gets us the most coverage with adaptive streaming for now.

    Take a look at the following on dynamic packaging documentation to get going:

    http://channel9.msdn.com/Series/Windows-Azure-Media-Services-Tutorials/Introduction-to-dynamic-packaging

    http://msdn.microsoft.com/en-us/library/jj889436.aspx

    Tuesday, April 22, 2014 4:51 PM
  • Thank-you for your detailed response. After following the tutorials and re-encoding, I believe we have things closer to working. I was able to use some sample code to generate the URLs for HLS and Smooth streaming and the -v3 results in a manifest being downloaded. 

    The HTML5 version works right in the browser as one would expect. JWPlayer still does not work, says file is not available for offline viewing. 

    Smooth streaming works using  a Silverlight Test Player http://playready.directtaps.net/pr/doc/slee/ with the manifest located at http://trekmediasvcdevuseast01.origin.mediaservices.windows.net/49890289-3ed2-435e-8102-5187b9dab11a/domane_tech_web_en.ism/manifest

    jwplayer is complaining that "offline playback is not supported". Seems like a jwplayer issue rather than a media services issue since smooth streaming works. 

    http://trekmediasvcdevuseast01.origin.mediaservices.windows.net/49890289-3ed2-435e-8102-5187b9dab11a/domane_tech_web_en.ism/manifest(format=m3u8-aapl-v3)

    Tuesday, April 22, 2014 8:50 PM
  • Yeah, that is because the flash player doesn't work in a local file.  You can host it locally in an ASP.NET web server, though.

    This is what they say in the JW Player docs:

    This means the page with JW Player embedded is not loaded from a webserver.  Due to various online-offline security restrictions, such a setup is not supported in the Flash rendering mode.  Either upload your page to a webserver or run a local webserver to circumvent this."

    For the heck of it, I threw a link up to your HLS V3 version on our website in a test page.  Playing great!

    http://mail.yogadownload.com/test-trek-hls.html

    Justin

    • Proposed as answer by reklats Tuesday, April 22, 2014 9:54 PM
    • Marked as answer by JG - YDL Tuesday, April 22, 2014 10:03 PM
    Tuesday, April 22, 2014 9:26 PM
  • Seems to be a JW Player configurator issue. I was able to repro the same problem on the hosted JWPlayer site when attempting to use their configuration tool.

    Let me ask the folks over at LongTail what's going on there in their player tool and then update the thread. 
    Justin is right though, if you download the player source and host it yourself it appears to work just fine.

    Tuesday, April 22, 2014 9:31 PM
    Owner
  • Got an answer back fast.

    Eric at JWPlayer said that you just need to add the following to your embed code.

    type:"HLS"

    Their hosted JWPlayer publishing tool at jwplayer.com is not automatically doing that for our URL format. I put in a request for them to support that. For now you can use the tool to configure the player, grab the embed code and script tag and put it into your web page and then update the type:"HLS" property to make it work. 

    The reason it is not working is because we use a RESTful URL for our manifest files, and we don't put any extension in the file URL.  It messes up some players that parse the URL for .msu8 extensions.

    Tuesday, April 22, 2014 9:52 PM
    Owner
  • Thanks for all the help and for hosting the test page. We finally got it working.
    Tuesday, April 22, 2014 9:54 PM
  • Cool.  Yeah I forgot about that, we have been using type: "m3u8".

    Justin

    Tuesday, April 22, 2014 9:55 PM
  • Blog post to follow up on this thread:

    http://blog-ndrouin.azurewebsites.net/hls-v3-new-old-thing/

    MSDN doc updates within a few days.

    Thursday, May 8, 2014 11:55 PM
    Owner