The following forum(s) have migrated to Microsoft Q&A (Preview): Developing Universal Windows apps!
Visit Microsoft Q&A (Preview) to post new questions.

Learn More

 locked
[UWP]Background audio with MFT in Windows 10 RRS feed

  • Question

  • Hey,

    I'm trying to write a new Windows 10 version of my existing Windows 8.1 App.
    However it seems like you changed a lot regarding audio.
    A Media Element doesn't work in background anymore or at least I didn't get it to work.
    I'm open for suggestions.

    From what I've found I need a Background Audio Task (the documentation is pretty right now so I'm not 100% sure that I understood how to implement this).
    However the MediaPlayer unlike the MediaElement doesn't seem to support Audio Effects.
    Is there any way to apply a Media Foundation Transform?

    Kind regards,
    Stefan

    Friday, July 31, 2015 7:38 PM

All replies

  • Hi Stefan,

    >> A Media Element doesn't work in background anymore or at least I didn't get it to work.

    As I saw from the Visual Studio, the AudioCategory.BackgroundCapableMedia is obsolete.


    >> A Media Element doesn't work in background anymore or at least I didn't get it to From what I've found I need a Background Audio Task (the documentation is pretty shitty right now so I'm not 100% sure that I understood how to implement this).

    In the Windows Universal Sample package, there is a sample named “Background Audio” which demonstrated how to play audio in the background with the BackgroundMediaPlayer APIs.

    https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/BackgroundAudio

    >> However the MediaPlayer unlike the MediaElement doesn't seem to support Audio Effects.

    Currently, I did not find the API to add effect in BackgroundMediaPlayer, I suggest you submitting a feature request to the user voice.

    Regards,

    Jeffrey


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Monday, August 3, 2015 7:44 AM
  • Thank you, I already found that and implemented a BackgroundAudioTask.
    Though the sample was only little helpful since it doesn't run.

    I added a feature request. However I don't have much hope of this getting implemented in the near future.
    People asked for it on the WP Version of BackgroundMediaPlayer already a year ago and nothing happened.
    It's kinda sad how the almighty Windows 10 UAP API offers less features than Windows 8.1 in that aspect.
    Monday, August 3, 2015 8:08 AM
  • It didn't make it through because it wasn't really needed.

    You have the media stream source class which allows you to do pretty much everything with the rendering pipeline of the system. It isn't easy, but it is far easier than what you would face when dealing with MFT.

    Monday, August 3, 2015 8:40 AM
  • I'm not sure if I understand what you mean. How would I add a Media Stream Source to the BackgroundMediaPlayer pipeline?

    Well I've already written the MFT so I doubt there is an easier way than calling AddAudioEffect.
    Monday, August 3, 2015 8:53 AM
  • SetMediaSource...

    Writing a media stream source is way easier than a MFT :P

    Tuesday, August 4, 2015 5:50 AM
  • Yeah, thank you. I'm not stupid ;)
    But doesn't a MediaStreamSource only take a stream as input?
    How do I get the raw audio data without having to write my own less efficient and way more battery draining decoder?
    Also it seems like you have to write it in C++ since the C# interfaces are empty, is that correct?
    Any resources on how to implement that?

    PS: Using an existing MFT is way easier than writing a new MediaStreamSource :P
    Tuesday, August 4, 2015 8:16 AM
  • You can do whatever magic you want with the streams. As long as you set the proper encoding, the audio pipeline doesn't care how you get the samples.

    The battery drain is insignificant. I've moved away from hardware accelerated decoding in my app, and battery drain stayed pretty much the same.

    What interfaces are you talking about? The media stream source class implements the IMediaSource interface directly. You don't have to worry about it. In fact, a media stream source is quite a low level object.

    You aren't forced to write the media stream source adapter itself in C++, only the decoder must be C++. But you will run into a bit of trouble if you choose the C# way.

    http://social.technet.microsoft.com/wiki/contents/articles/30827.implementing-media-stream-sources-in-windows-phone-runtime-for-background-audio-tasks.aspx

    Tuesday, August 4, 2015 8:33 AM
  • thank you I'll try that later :)
    Tuesday, August 4, 2015 8:46 AM
  • This is extremely disppointing!! Oh man.

    So you are suggesting we implement all audio codecs ourselves? You can't be serious. How is that goint to be easier. Implementing an MFT is really not that hard, and can be used with any codec available. How could they deprecate MediaElement background audio and not allow MFTs on BMP?

    One more item in the long long list of Windows 10 disappointments.

    Tuesday, August 4, 2015 10:40 AM
  • Depends on what you want to do.

    I don't think background capable media element is truly deprecated. That warning serves as a ...well ... a warning to make sure you understand that it is not going to work on all platforms (as in it won't work on phones).

    Why? because the phone background audio architecture is different than the one from desktops. Mainly because on desktops, the audio task is actually attached ultimately to a window, whereas in windows phone, the audio task is a process all in itself.

    If you want to do MFTs on desktops only, feel free to use media element. This isn't going anywhere. Phones are a bit more complicated, because whether we want to admit or not, the form factor and performance does make a difference.

    • Edited by mcosmin Tuesday, August 4, 2015 11:53 AM
    Tuesday, August 4, 2015 11:48 AM
  • Nope doesn't even work on desktops anymore since the sound is muted as soon as you minimize the app.
    Tuesday, August 4, 2015 4:42 PM
  • Sorry about the difficulty here. Let me clarify a few points.

    If you have a desktop only app and you liked AudioCategory.BackgroundCapableMedia then you can keep using it for now. There is a compile-time warning but it still works with Windows 10. The Groove Music app currently uses this approach, so let's figure out why it's not working for you.

    If you're observing muting this is most likely due to missing one of the requirements on this Windows 8.1 page.

    https://msdn.microsoft.com/en-us/library/windows/apps/xaml/jj841209.aspx

    Specifically, you must add the following to your appxmanifest:

    <Extension Category="windows.backgroundTasks" StartPage="MusicApp\MusicApp.html">
      <BackgroundTasks>
          <Task Type="audio" />
      </BackgroundTasks>
    </Extension>
    

    You must set an AudioCategory on MediaElement:

    <MediaElement Name="media" 
                  AudioCategory="BackgroundCapableMedia" 
                  Source="Somesong.mp3" />

    You must support the SystemMediaTransportControls:

    SystemMediaTransportControls systemControls;
    
    void InitializeTransportControls()
    {
        // Hook up app to system transport controls.
        systemControls = SystemMediaTransportControls.GetForCurrentView();
        systemControls.ButtonPressed += SystemControls_ButtonPressed;
    
        // Register to handle the following system transport control buttons.
        systemControls.IsPlayEnabled = true;
        systemControls.IsPauseEnabled = true;
    }

    The deprecation was put in place because we prefer that folks writing new apps, or who those with existing apps who are able, migrate to the more memory friendly UWP model that is available everywhere.

    If there are specific reasons you can't or don't want to do that, we would appreciate the feedback.

    This thread's feedback is a great example. We don't have support yet for audio effects in our background media player which leaves MediaStreamSource as the only real option when using that model. That's a clear gap from what we provided with the Windows 8.1 desktop model and we are reviewing the possible options to enable support.

    Wednesday, August 19, 2015 12:53 AM
  • Hi Aaron, thanks for the update on this. It looks like this does in fact work for many Windows 10 users, but for others it does not. I am using the MediaElemet approach in my app. Since the app could get suspended on track end even on Win81, I am using two MediaElements and switch them on track end. This technique was suggested in one of the MSDN pages about background audio. This works very well in Windows 8.1. Of course, I also have the capability declared, otherwise this would not work on Win81 as well.

    But now as I said, some of my users report that on Windows 10, the app gets suspended on track end, and they have to re-open the app to get the next track running. This is of course a very bad situation and makes my music player app effectively useless for all affected users. In other forum topics, other developers have reported the same. It looks like a lot of background audio apps that worked well on Windows 8.1 suddenly get suspended on track end, at least for some Windows 10 users. It is the same app, same bits that run well on Windows 8.1. I do not (yet) have a separate build for Windows 10.

    It would be great if your team could look into this and provide information what can be done, to prevent this from happening. Please let me know if you need more input on this.

    Wednesday, August 19, 2015 6:29 PM
  • Lukas, thanks for reporting this. Let me check in with the team to see whether this has been reproduced. If there are threads with additional details I should look at as well please point me at those. I would also be interested in reviewing that MSDN page recommending switching between two MediaElements if you have that link handy.

    In the meantime, would it be possible for you to test an alternate approach?

    We have a new MediaPlaybackList specifically for multi-track scenarios and I would be curious to know whether you see this problem when letting the platform manage track changes through that list.

    The other advantage to using that list is it supports gapless playback if your content has been encoded with the appropriate metadata (like live concerts) and makes a best effort otherwise.

    The VideoPlayback and BackgroundAudio samples show the usage.

    Wednesday, August 19, 2015 8:50 PM
  • Also, if you know of a store app that reproduces this we can do some debugging on this end that may tell us why the app is getting suspended. Or, if you have a basic repro project to share that would help too. :-)
    Wednesday, August 19, 2015 9:22 PM
  • As I said, I am not able to reproduce this myself.

    The switching MediaElement technique is actually the only way to get reliable background playback on Windows 8. Even Windows 8 was overly aggressively suspending background audio apps as soon as playback ends. It would be a lot better to just wait a few seconds and then check again if really no more playback happens, before suspending. When you minimize an app, it also does not get suspended right away, for good reason.

    Links on this:

    https://social.microsoft.com/Forums/en-US/ae344adb-8314-4f0f-98c2-deb15dbed830/app-gets-suspended-while-playing-audio-in-background-when-in-connected-standby?forum=winappswithcsharp

    https://social.msdn.microsoft.com/Forums/windowsapps/en-US/a90fcc68-6cbf-4991-9e38-6f43563f10e4/background-audio-during-connected-standby?forum=winappswithcsharp

    https://social.microsoft.com/Forums/en-US/4af53ba0-906d-4f99-9d70-a91ed7a27c0b/mediaelementmediaended-does-not-fire-when-audiocategory-audiocategorybackgroundcapablemedia?forum=winappswithcsharp

    https://msdn.microsoft.com/en-us/library/windows/apps/hh452805.aspx?f=255&MSPPError=-2147217396

    Someone else with this issue on Win10:

    https://social.msdn.microsoft.com/Forums/en-US/e3033fb4-e24f-47fc-a695-71e96ae20682/w81problem-with-apps-sound-on-windows-10?forum=wpdevelop

    My assumption is that Windows 10, instead of giving the app more time, suspends the app even a tad faster whan Win8, so now even the switching technique does not guarantee reliable background playback anymore. I really hope tha this can be fixed in a Windows 10 update, because most probably a lot of media players built for Windows 8 are affected, most just don't know yet.


    • Edited by Lukas F Thursday, August 20, 2015 4:17 AM
    Thursday, August 20, 2015 4:16 AM
  • I personally never heard of this switching media element technique and I am pretty sure you don't actually need it. If you are doing everything by the book, you don't need more than 1 element. Your mistake is probably somewhere in the event listening of your media element. You most likely failed to attach or properly implement an event handler and the app gets disconnected on track end.

    I have implemented both windows 8.1 and windows phone 8.1 background audio and never had a problem with track switching, and nor have my users. If there was ever an issue, it was something that I did and not the platform.

    Thursday, August 20, 2015 6:22 AM
  • Thank you for those links. We'll investigate.

    Apps are supposed to have a grace period of approximately 20 seconds.

    The challenge we have is that we're not able to reproduce this with our sample app and so we need a good example to debug.

    Since you can't reproduce this either, do you have any customers who are able to reproduce this consistently that you could put us in touch with?

    You can email me at "aaron.oneal". You know the domain and hopefully the spambots don't. :)


    Thursday, August 20, 2015 6:31 AM
  • Thanks Aaron, I will see if I can get you in touch with one of the customers.

    @mcosmin: Interesting to hear that for some apps it works with only one ME (on WP81, you surely did not use an ME because that does not allow background audio). But from the links you can see that it's not only me who is affected. And these are only a few quick searches, there are more if you search more. Be assured that I am able to implement and attach an event handler, since I am doing C# software development professionally since more than 10 years. Also, the next track is instantly played once the app gets activated, this clearly shows that the MediaEnded event handler was in fact attached properly and did set the next track. It's just that the app was suspended during that operation. Back then I also added logging of course and saw that the next track was already set on the ME, SetSource was called, but still playback would only start when the app was activated next time. So the app was suspended while the ME was just loading the next file. And SetSource was called few ms after MediaEnded. Also, it usually would work for a while even in background, but after a while (sometimes after a few tracks, sometimes after 1-2h of playback) it would just stop working, until unlocking the device and showing the app once. It happened a more often on low end devices such as my Atom based tablet. That's when I looked for help in the forums and found lots of others affected, and the resolution from Media team to use two switching MEs, which did work for me and all other affected devs.

    Thursday, August 20, 2015 2:39 PM
  • After contacting one of the affected users, it looks like I described the problem incorrectly. The next track does in fact start and is playing, but for some reason, it is not audible. As soon as the users moves the seek bar slider, sound gets audible again. So it probably does not have to do with app suspension but is a different problem. And now it actually sounds a lot more like the problem the other dev reported (last of my links above). I wrote a mail to bring you in contact with the affected user.
    Thursday, August 20, 2015 5:17 PM
  • :)

    are you using any custom made decoders, MFTs or MSSs?

    Friday, August 21, 2015 6:11 AM
  • This topic is about background audio with MFTs (Audio Effects). The BackgroundMediaPlayer does not let us add AudioEffects, so it is not possible to realize equalizers or other effects. So this does not answer the question.

    AudioEffects for BMP have been requested on UserVoice long ago for WP81 already. Pretty disappointing that this has not been realized, even though BMP is now also the recommended way for background audio on PC.

    Wednesday, October 21, 2015 11:10 AM
  • This topic is about background audio with MFTs (Audio Effects). The BackgroundMediaPlayer does not let us add AudioEffects, so it is not possible to realize equalizers or other effects. So this does not answer the question.

    AudioEffects for BMP have been requested on UserVoice long ago for WP81 already. Pretty disappointing that this has not been realized, even though BMP is now also the recommended way for background audio on PC.


    He is spamming. Ignore him.
    Wednesday, October 21, 2015 1:39 PM
  • Hi folks, here's the latest--

    We determined that the previously reported muting problem was due to the way certain apps were swapping elements using DispatchTimer which we freeze in background scenarios. We've made a change to this behavior for future releases that should allow apps to use that technique although in many cases we recommend MediaPlaybackList as a better option (and it supports gapless audio transitions).

    We have also heard the feedback that background audio effects are important for certain customers and we have enabled this as well.

    Windows Insiders have access to the latest builds and we appreciate any feedback you can provide on those through the Insider forums and Feedback app. Thank you!

    Wednesday, October 21, 2015 5:45 PM
  • so in windows 10 we can use MFTs with background media player?
    Wednesday, October 21, 2015 7:29 PM
  • Thanks for the update Aaron! Good to hear that you will be addressing both issues. So am I right in assuming that both will come with the TH2 update (scheduled for november)?
    Wednesday, October 21, 2015 7:58 PM
  • Please refer to the Windows Insider Program for pre-release details and a look at what's coming.
    Wednesday, October 21, 2015 8:40 PM
  • Hi Aaron -

    Minor tangent to the original thread... but I'm hoping you can help. I have a Win10 app which uses the deprecated approach you've mentioned. It generally works fine, except that at seemingly random times playback will start (e.g. when the app isn't running and sometimes when you're not even using the PC). Any suggestions on why this may be happening?

    Tom

    Saturday, November 21, 2015 2:35 AM
  • Hi Tom,

    I don't recall any other reports about this. The only thing I can think of is the app is running (use Task Manager to verify) and the volume control might be starting playback again. Please let me know if you're able to narrow a more specific set of repro steps and we can dig deeper.


    Saturday, November 21, 2015 3:00 AM