Skip to main content

 none
Redirected audio streams faster than associated video resulting in unwanted large decoded buffer build up RRS feed

  • Question

  • Hi all,

         I've worked from WavSink samp
    le code which emits a decoded audio bytestream in chunks.

    Rather than save to a fi
    le as per the sample's original purpose, I direct the PCM stream at an audio engine (FMOD).

    The audio engine consumes buffers at rea
    l-time rate as it plays, however WavSink sends buffers way ahead of when they're required resulting on a large buildup of decoded buffers that aren't yet needed and so are consuming excess amounts of memory.

    I've tried various approaches to manage the Media &
    Stream Sink output but can't get it quite right.

    My most successfu
    l does halt the audio stream intermittantly to 'hold it back', with a release to continue when I want more data, but this results in video glitching.

    I tried creating a seperate c
    lock for the audio to attempt to decouple it to address this. Got that working but it doesn't resolve the glitch.

    Best I've got so far is to queue up  IMFSample instances at stream Output, but that just pushes the buffer buildup further back - they're still there.

    Wanting a way to ask the audio Stream/Sink to stop sending me more data for a while when I have enough, then ask it to continue when I want some more without causing the video to glitch.

    Does anyone know how to achieve this, p
    lease.

    Cheers,
    Steve.



    Thursday, November 14, 2019 12:58 PM

All replies

  • Next experiment is to see if I can do this using a Transform, either instead of a sink or as a passthrough with some kind of flow control

    Clues welcome :)

    Thursday, November 14, 2019 5:13 PM
  • Tried a transform

    GetInputStatus(DWORD dwInputStreamID, DWORD *pdwFlags)
    {
     ...

      if (m_saturated)
            *pdwFlags = 0; 

    ...

    }

    ... Results in audio still being forced thru the pipe & discarded whilst I have saturated flag set rather than have it wait until I want some more.

    The media foundation black box is getting tiresome. Wouldn't mind if it worked or if support was good.

    Neither of which are true.

    Friday, November 15, 2019 4:20 PM

  • IMFQualityManager::NotifyProcessInput
    ?

    or will that just drop samples

    like transforms do ?

    Thanks :)


    Saturday, November 16, 2019 2:08 PM
  • Hi Steve,

    Based on the description, it seems that the application’s topology and sequence are  “Video or Camera (Media source)-> Media Session -> Media Sink -> FMOD (maybe https://www.fmod.com/core) -> audio renderer” .

    > The audio engine consumes buffers at real-time rate as it plays, however WavSink sends buffers way ahead of when they're required resulting on a large buildup of decoded buffers that aren't yet needed and so are consuming excess amounts of memory.

    To compose the samples that FMOD needs, you may need multiple media samples of MF. Finally, the media sink need amount of memory for keeping multiple samples.

    There is not how to change the timing without stop/start. However, if uses stop/start, video glitching will be occur.  Does the application make a queue and worker thread for FMOD? Media Sink insert a sample to the queue every time. Worker thread dequeue the samples and make buffer of FMOD Next, the worker thread should free the samples which is no needed.  

    Please feel free to let me know if I understood correctly. And if you are still have problem about it, please share a simple demo to reproduce/demonstrate this issue.

    Regards & Fei


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Thursday, November 21, 2019 4:45 AM