locked
[UWP][C#] Record audio with AudioGraph RRS feed

  • Question

  • Hi all,

    I'm writing an app which should analyze audio input. I was able to record audio data to a file, output that data from file and generating an own output (for example 440Hz). However, I was not able to capture the AudioFrames / quantum data without recording to a file. As far as I know, this should be possible with AudioFrameOutputNode (AudioGraph Documentation).

    Here is my Code:

    /// <summary>
    /// Initializes the AudioGraph
    /// </summary>
    /// <returns>true on success</returns>
    private async Task<bool> InitAudioGraph()
    {
        var audioGraphSettings = new AudioGraphSettings(AudioRenderCategory.Media);
        audioGraphSettings.PrimaryRenderDevice = null;
    
        var result = await AudioGraph.CreateAsync(audioGraphSettings);
    
        if (result.Status == AudioGraphCreationStatus.Success)
        {
            this.audioGraph = result.Graph;
    
            return true;
        }
    
        return false;
    }
    
    private async void StartRecord()
    {
        if (!await InitAudioGraph()) return;
    
        this.audioGraph.Start();
        this.frameOutputNode = this.audioGraph.CreateFrameOutputNode();
        this.frameOutputNode.Start();
        this.audioGraph.QuantumStarted += AudioGraph_QuantumStarted;
    }
    
    private void AudioGraph_QuantumStarted(AudioGraph sender, object args)
    {
        var frame = frameOutputNode.GetFrame();
        ProcessFrameOutput(frame);
    }
    
    private async void ProcessFrameOutput(AudioFrame frame)
    {
        using (var buffer = frame.LockBuffer(Windows.Media.AudioBufferAccessMode.Read))
        {
            ProcessAudioBuffer(buffer);
        }
    }
    
    unsafe private void ProcessAudioBuffer(AudioBuffer buffer)
    {
        using (var reference = buffer.CreateReference())
        {
            byte* dataInBytes;
            uint capacityInBytes;
            float* dataInFloat;
    
            // Get the buffer from the AudioFrame
            ((IMemoryBufferByteAccess)reference).GetBuffer(out dataInBytes, out capacityInBytes);
            dataInFloat = (float*)dataInBytes;
    
            if (capacityInBytes > 0)
            {
                var floatCapacity = buffer.Length / 4;
                float[] inputData = new float[floatCapacity];
    
                for (var i = 0; i < floatCapacity; i++)
                {
                    inputData[i] = dataInFloat[i];
                }
            }
        }
    }


    For some reason, each float in the inputData-array has 0 as value. When using equivalent code for writing the data to a file, everything is fine.

    First, I thought that the implementation of IMemoryBufferByteAccess did not work, but as I am able to generate audio output with this implementation it should also work with audio input.

    I would appreciate any help.

    Kind regards,

    Manuel Kurtz


    Manuel Kurtz

    Saturday, September 1, 2018 11:31 AM

Answers

  • Hi,

    After researching, I found the problem. You need to add outgoing connection between input node and output node. 

    I tried your code in official sample AudioCreation. I adjust the code in scenario2 to test your code

    In that scenario, if I do not establish connection between deviceInputNode and AudioFrameOutputNode, I'll get the same null value like you.

    If I get outgoing connection, the codes will get to data correctly

    frameOutputNode =  graph.CreateFrameOutputNode();
    graph.QuantumStarted += AudioGraph_QuantumStarted;
    deviceInputNode.AddOutgoingConnection(frameOutputNode);


    Best regards

    Roy


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Tuesday, September 4, 2018 2:45 AM

All replies

  • Hi,

    I tried this in Official sample and I could reproduce this issue. Thank you for reporting this. I'll consult other engineers for this.

    Best regards,

    Roy


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Monday, September 3, 2018 11:23 AM
  • Hi,

    After researching, I found the problem. You need to add outgoing connection between input node and output node. 

    I tried your code in official sample AudioCreation. I adjust the code in scenario2 to test your code

    In that scenario, if I do not establish connection between deviceInputNode and AudioFrameOutputNode, I'll get the same null value like you.

    If I get outgoing connection, the codes will get to data correctly

    frameOutputNode =  graph.CreateFrameOutputNode();
    graph.QuantumStarted += AudioGraph_QuantumStarted;
    deviceInputNode.AddOutgoingConnection(frameOutputNode);


    Best regards

    Roy


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Tuesday, September 4, 2018 2:45 AM
  • Hi,

    Thanks a lot for this explanation. I must have overseen this line which changes everything.

    Kind regards,

    Manuel


    Manuel Kurtz

    Wednesday, September 5, 2018 4:17 PM