The following forum(s) are migrating to a new home on Microsoft Q&A (Preview): Developing Universal Windows apps!

Ask new questions on Microsoft Q&A (Preview).
Interact with existing posts until December 13, 2019, after which content will be closed to all new and existing posts.

Learn More

 none
[WP8.1][C#] Poor audio quality in recorded video using MediaCapture APIs RRS feed

  • Question

  • Hello

    I have been using my own app for long months already.

    My app is a custom camera app, which captures the video from the front or the back camera of the device. I have implemented this functionality as shown in the MSDN example:

    https://msdn.microsoft.com/en-us/library/windows/apps/xaml/Dn642092%28v=win.10%29.aspx

    However, the audio quality of recorded video is extremely poor when compared to video recorded in the exact same light and ambient sounds conditions, but using eg. Lumia Camera or built-in Windows Phone camera app.

    While the built-in Camera app records a mp4 video file with crisp, clear and overally VERY high quality sound, MediaCapture produces mp4 video file with noticeably worse sound quality. It sounds like if it was encoded in 64kbits mp3 (some kind of "underwater" and "metallic" sound).

    I have tried various workarounds trying to make the recorded sound better, for example I manually tried to set higher bitrate of Audio property in MediaCaptureProfile instance, like this:

    mediaCaptureProfile = MediaEncodingProfile.CreateMp4(VideoEncodingQuality.HD1080p);
    mediaCaptureProfile.Audio.Bitrate = (uint)341000;

    Setting ANY Bitrate value except 192000 immediatealy crashes the app when the recording is started with this exception:

    "The data specified for the media type is invalid, inconsistent, or not supported by this object."

    I also tried manually setting the AudioDeviceId during MediaCapture initialization (just like VideoDeviceId from the example). But this made the app to be less stable and it crashes on longer recordings (longer = more than 20 seconds). Moreover, the sound quality is the same.

    Is it possible to force the MediaCapture to encode the sound in better quality? It MUST be possible to achieve, since the Stock Camera app or Lumia Camera apps records really perfect audio quality in every recorded video. I use my app to record my band rehearsals or live shows so audio quality is quite important factor for me and I am forced to use my own app (not Lumia Camera or the stock one) as my app performs some other tasks supporting us while playing.

    Many thanks in advance for any help or tip :)

    Best regards

    Monday, October 12, 2015 10:07 PM

Answers

  • Ok, I've found a solution.


    mediaCaptureProfile = MediaEncodingProfile.CreateMp4(<any desired enum>);
    mediaCaptureProfile.Audio = GetAudioEncodingPropertiesFromAutoVideoEncodingQuality();
    
            private AudioEncodingProperties GetAudioEncodingPropertiesFromAutoVideoEncodingQuality()
            {
                var encodingTemp = MediaEncodingProfile.CreateMp4(VideoEncodingQuality.Auto);
                return encodingTemp.Audio;
            }

    Works like a charm and succesfully workarounds the bug :).

    Thanks for help

    Regards



    • Edited by mmaciekksz Sunday, November 1, 2015 8:08 PM
    • Marked as answer by mmaciekksz Sunday, November 1, 2015 8:08 PM
    Sunday, November 1, 2015 8:08 PM

All replies

  • Which device are you using? based on my experience, this is related to your device, on my Lumia1520, the audio quality is good

    Best Regards,
    Please remember to mark the replies as answers if they help

    Tuesday, October 13, 2015 9:46 AM
  • I've tested it on Lumia 635, 735 and 830.

    All had exactly the same issue with sound quality. The high frequencies (that is the drum cymbals hits) are extremely poor. They sound as rly 64 kbits mp3. Deep underwater and metallic sound is heard.

    As I told - we are recording music band rehearsals. Quiet sounds are ok.

    And one more thing - my app has option to record audio only to wav file. When this is selected (no video is captured) - the sound quality is just perfect.

    Is it possible to encode such great sound quality in mp4 video? Lumia Camera and built-in app can easily handle it since mp4's sounds captured using those apps are just brilliant.

    Tuesday, October 13, 2015 7:32 PM
  • What AudioDevice are you using in your MediaCapture initialization settings? Try using the one passed by the Windows.Media.Devices.MediaDevice.GetDefaultAudioCaptureId API.
    Tuesday, October 20, 2015 12:34 AM
  • Hello


    Many thanks for your response. My app has a feature to select one from available audio capture devices to check if the selection has any impact on the sound quality. On my Lumia 735, some choices records the silence, others crash the application. 2 of 4 were working, but the sound quality is exactly the same :(.


    I also tried by just ommiting the AudioDeviceCaptureId. The sound quality remained the same.

    One more important thing I forgot to mention. I call this on my MediaCapture instance:

    mediaSettings.MediaCategory = MediaCategory.Communications;

    Is it possible to force the system to encode the audio with higher bitrate? Or any possible solution to make the recorded sound be better :(?

    Many thanks in advance

    Regards

    Wednesday, October 21, 2015 8:37 PM
  • Could you try setting it to MediaCategory.Media, or even leaving it out completely and just using the default value? Because Communications is described as "Media is intended for real-time communications", I wouldn't be surprised if this were the reason you have lower quality audio.
    Wednesday, October 21, 2015 9:01 PM
  • Hello

    Well, Communications is the only one enum value, which allows me to achieve my requirement for the app functionality.

    However, I can try set it to different value, but I am pretty sure it won't help. My app also records a WAV audio using the Communications media category too and the sound quality is perfect.


    • Edited by mmaciekksz Sunday, October 25, 2015 12:29 PM
    Sunday, October 25, 2015 12:29 PM
  • You mentioned that if you only record audio, then the quality is good, but if you record video with sound, then audio quality suffers.

    Can you try the CameraStarterKit sample and see if it reproduces your issue? It targets Windows 10, but for the most part, you should be able to just copy & paste the code into a Windows 8.1 project.

    At the same time, would you mind sharing the smallest possible snippet you can come up with, that will reproduce the issue?

    Tuesday, October 27, 2015 12:03 AM
  • Hello

    Many thanks for your response :).

    I will try the snippet you provided on my phone and I will post here about the results next week.

    About the snippet. I could post some of my code, but believe me - it is mostly copy paste from the MSDN tutorial I posted above. The one exception is the Media Category I set additionally and the internal, complex app logic to e.g. determine if WAV or MP4 file should be recorded.

    I will deeply analyze the code you provided and compare with mine if the recording results will be good enough.

    Best regards

    Tuesday, October 27, 2015 9:56 PM
  • No problem! If you can make one of our camera samples reproduce the issue on your devices (without any modifications), we'll have to dig in deeper. At that point I'd like you to share the project to speed things up.

    I'll stay subscribed to this thread to get notified when you have more information to share.

    Wednesday, October 28, 2015 4:10 AM
  • Hello

    Many thanks for your response :).

    I will install the sample app you provided on my old Lumia 635 today. I will be in my rehearsal room next thursday, so I am going to record a sample drum beat on both apps (mine and the sample you provided), as loud as possible. If no differences are heard on both recordings (both would be poor), I will extract the audio files from mp4s recorded and post it here to allow you investigate deeper.

    Best regards

    Wednesday, October 28, 2015 11:33 AM
  • Hello

    As I promised - I did some test recordings on my rehearsal room today. I made 2 recordings using the sample app and one using my app.

    Fortunately, the sample app makes a video with good sound quality. I also modified the sample you provided by setting the Media Category to Communications and the audio was good there, too.

    I am now a bit confused, what is going on and why is that. I will carefully check out my code and compare it to the sample. Since I am busy, this may take me some time, but I will surely post my thoughts here when I am done.

    Many thanks for help provided :)

    Best regards

    Thursday, October 29, 2015 10:19 PM
  • The important thing is that now you have a way to capture high quality audio! :)

    I hope the sample allows you to understand why your own project wasn't working so well. I'm proposing my post with the link to the sample as an answer, but it would be great if, when you figure it all out, you came back to let me know what the problem was.

    Happy developing!

    Thursday, October 29, 2015 11:06 PM
  • Hello


    I managed to have a look on the sample code you provided. I didn't even have to compare it line by line to my code to notice the first difference:


    The sample code sets the MediaEncodingProfile for video file like this:

    var encodingProfile = MediaEncodingProfile.CreateMp4(VideoEncodingQuality.Auto);

    My app chooses the VideoEncodingQuality automatically based on user's selected recording settings, which my app allows to change. On my phone and my settings those are HD1080p or HD720p. What I did is I changed the above line to this one in the sample app:

    var encodingProfile = MediaEncodingProfile.CreateMp4(VideoEncodingQuality.HD720p);

    And v'oila! The crap, metallic, 24kbits-like sound in video recording came back :). To test it on your own without a need of playing drums or any other musical instrument I suggest to record two videos using the sample app you provided with as loud as possible music playing (put the phone 1-2 cm next to the speakers while recording, the more powerful speakers the better). First, set the VideoEncodingQuality to Auto, then to HD720p or HD1080p (I've tested only those enum values so far, but I am pretty sure any other would reproduce the bug). Then compare the recordings. I am sure you will hear the difference.

    At this point I am pretty sure I have found an evidence for a bug in MediaCapture APIs. I am now going to manually set this enum value to Auto, but I want my app to allow users select their own settings. And by the way - I honestly do not know, how Auto enum works and what video resolution and bitrate is chosen. I would be pleased if you explained this :).

    Please let me know if you can suggest me any workaround. I CAN temporarily set the enum to "Auto", but it is really quite important for me to set this dynamically based on user's settings.

    Also, please let me know if there is any chance to fix this OS issue on Windows Phone 8.1, without waiting for official release of Windows 10 Mobile.

    Best regards

    Friday, October 30, 2015 2:07 PM
  • Ah, so you want to offer different video resolutions to your users. Because that is a slightly more complicated scenario, I'll have to refer you to the CameraResolution SDK sample on GitHub and this answer on StackOverflow. Still, I'd like to give you a little insight here.

    What maybe isn't so clear to you right now, is that the camera API has a pipeline model, where there is a source that produces frames (think of this as the sensor) and there is a sink that saves the frames (think of this as the component writing everything out to e.g. a file). By setting the encodingProfile to a specific resolution, you're effectively transcoding frames produced by the sensor at whatever its default resolution is, into e.g. 720p or 1080p. What you really want, for maximum efficiency and quality, is to configure the source to produce frames at the desired resolution, and then have your encoder (or sink) set to "passthrough" mode (VideoEncodingQUality.Auto).

    I'm sorry our Camera documentation on MSDN hasn't covered this scenario yet, but we're working on that. In the meantime, the camera samples on GitHub should be your best friends. :)

    Friday, October 30, 2015 4:55 PM
  • Hello

    Thank you for the explaination and the link with example, but I am afraid we have misunderstood :).

    My app is doing the exact same thing as shown in CameraResolutionSDK sample. I manually set video resolution, framerate and the bitrate using MediaCapture.VideoDeviceController.SetMediaStreamPropertiesAsync. During the process, I do NOT touch the AudioDeviceController (however - I have tried many times to force the system to encode the audio in better quality, but any changes made there thrown an exception).

    Maybe I was not clear enough, but as I mentioned - setting ANY other VideoEncodingQuality enum than

    MediaEncodingProfile.CreateMp4(VideoEncodingQuality.Auto);

    produces the crap sound. Even though two lines ahead I do

    MediaCapture.VideoDeviceController.SetMediaStreamPropertiesAsync

    I think that video resolution, aspect ratio and frame rate have nothing to do with the audio quality, which is always crappy, 24-kbits mp3 like, when VideoEncodingQuality enum is chosen to any different value than Auto. I think this is an evidence of a bug in the APIs.

    Now I will try to extract the AudioDeviceController instance produced by VideoEncodingQuality.Auto and then assign it to AudioDeviceController produced by eg HD1080 enum value. I'll post my findings here.

    Meanwhile could you please tell me, what does the

    VideoEncodingQuality.Auto

    do under the hood? What will be the final framerate, resolution and aspect ratio of produced mp4 file when laucnhed on different phones with different cameras? How can the end user have impact on that (eg. will changing the system's default camera app settings have impact on the file produced)?

    Thanks in advance

    Regards

    Sunday, November 1, 2015 3:26 PM
  • Ok, I've found a solution.


    mediaCaptureProfile = MediaEncodingProfile.CreateMp4(<any desired enum>);
    mediaCaptureProfile.Audio = GetAudioEncodingPropertiesFromAutoVideoEncodingQuality();
    
            private AudioEncodingProperties GetAudioEncodingPropertiesFromAutoVideoEncodingQuality()
            {
                var encodingTemp = MediaEncodingProfile.CreateMp4(VideoEncodingQuality.Auto);
                return encodingTemp.Audio;
            }

    Works like a charm and succesfully workarounds the bug :).

    Thanks for help

    Regards



    • Edited by mmaciekksz Sunday, November 1, 2015 8:08 PM
    • Marked as answer by mmaciekksz Sunday, November 1, 2015 8:08 PM
    Sunday, November 1, 2015 8:08 PM
  • Hello

    Taking the opportunity, I would like to ask one more question regarding to audio quality with recordings.

    Is it possible to select a microphone to use to capture an audio stream from? I mean - to choose for example only the front microphone of Lumia 735, without using the back one next to the rear camera?

    I used the

    await DeviceInformation.FindAllAsync(DeviceClass.AudioCapture);

    and then set one of the IDs returned to proper AudioDeviceId. However, I have tried all of the values returned, but it seems that all of them use both microphones of my Lumia 735.

    Many thanks in advance

    Regards

    Monday, November 2, 2015 6:05 PM
  • Thanks for the follow-up, I will investigate the audio quality discrepancy you report to see if this is a bug or possibly undocumented behavior.

    As for your last question, I'm afraid that in the context of universal apps, the answer could vary depending on the driver of the device running your app. On phones that launched before Windows 10, but running the Windows 10 operating system, the audio system controls which endpoint is used. The endpoint you activate on will likely be chosen (especially if your application is the only application capturing in the system), however it is not guaranteed to be used and depends on what else is happening at the time. This is because only one endpoint can be used at a time on this hardware, and the audio system has to balance that choice across all audio clients. Your audio capture will still succeed, there will be no errors or indications otherwise, however the audio may not come from the specific endpoint chosen.

    Wednesday, November 4, 2015 2:10 AM
  • Ok, thank you.

    Please post here if you finish your investigations with the results found :).

    Regards

    Thursday, November 5, 2015 6:44 PM