locked
Bg Audio- Audio from .wav to Audio from .mp4

    Question

  • Hi Guys:

    I've currently working on a project which require the functionality of playing an background audio from a .wav file with a video from a .mp4 file. I am using media foundation to do this. I've realised that I would need to implement some sort of mixer to mix the IMFsamples from the audio stream of the video and audio file. I've also got the mixing algorithm to work as well. Currently this works if I grab backgroun audio from 2 different .mp4 files. Essentially discarding the video samples of the second .mp4 file and only using the audio samples.

    If I specify a .wav file for the background audio I start running into problems with different sized buffers from the IMFsamples. Where the samples from the .wav file contains buffers larger than the buffers from the the .mp4 file samples. This causes some samples to be dropped and produces poping audio playback.

    So what I'm confused about is that why are the buffer sizes still different even thought they have the same sample rate and bit depth?

    And what are the exact correct steps in mixing audio samples with different buffer sizes? Or are there ways to split the larger buffers into smaller sized buffers which can be mixed properly?

    Thanks

    Danny

    Monday, June 10, 2013 1:53 AM

Answers

  • Hello Danny,

    It sounds like you are trying to "mix in place". You should to create your own set of mix buffers and pass them instead of trying to use existing buffers. This will resolve your problem. I would recommend a pool of about 5 buffers to be safe. The duration of the audio mix buffers should be approximately the same as the display duration of each video frame.

    I hope this helps,

    James


    Windows SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

    Wednesday, June 12, 2013 11:06 PM
    Moderator

All replies

  • I've got a solution which reuses the longer IMFsamples, so I only have to use a Offset to mix it with each of the shorter IMFsamples. But this creates another problem of what happens when the shorter IMFsample is mapped to the end of the longer IMFsample and there isn't enough information needed...

    I've made an image to show my problem butm y account isn't verified yet by microsoft so I can't attach the image. You can see it at http://postimg.org/image/6knlp7x6p/

    Monday, June 10, 2013 2:09 AM
  • Hello Danny,

    It sounds like you are trying to "mix in place". You should to create your own set of mix buffers and pass them instead of trying to use existing buffers. This will resolve your problem. I would recommend a pool of about 5 buffers to be safe. The duration of the audio mix buffers should be approximately the same as the display duration of each video frame.

    I hope this helps,

    James


    Windows SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

    Wednesday, June 12, 2013 11:06 PM
    Moderator