locked
Processing samples from an MP3 file

    Question

  • I need some help in order to process samples from an MP3 file. Note that I do not want to play it back or re-encode it into another format. What I need is buffers of uncompressed samples in PCM format so that I can perform processing on it. Also, I don't want it to be clocked, but I want to process the samples on demand (faster than realtime).

    Unfortunately the WinRT documentation is extremely lacking, so I've been having a hard time connecting all the dots. Below is some sample code showing what I want to do

    private async Task ProcessMusicFile(StorageFile musicFile) {

        var fileStream = await musicFile.OpenAsync(FileAccessMode.Read);

        var fileProps = await m_file.Properties.GetMusicPropertiesAsync();
        var retrieveProps = new List<string>();

        retrieveProps.Add("System.Audio.SampleRate");
        retrieveProps.Add("System.Audio.ChannelCount");
        retrieveProps.Add("System.Audio.EncodingBitrate");

        var encodingProps = await m_file.Properties.RetrievePropertiesAsync(retrieveProps);

        var encodingSampleRate = (uint)encodingProps["System.Audio.SampleRate"];
        var encodingChannelCount = (uint)encodingProps["System.Audio.ChannelCount"];
        var encodingBitRate = (uint)encodingProps["System.Audio.EncodingBitrate"];

        var mp3Props = AudioEncodingProperties.CreateMp3(encodingSampleRate, encodingChannelCount, encodingBitRate);
        var mp3Descriptor = new AudioStreamDescriptor(mp3Props);

        var mediaStream = new MediaStreamSource(mp3Descriptor);
               
        mediaStream.CanSeek = true;
        mediaStream.Duration = fileProps.Duration;

        var profile = MediaEncodingProfile.CreateWav(AudioEncodingQuality.Medium);
        var transcoder = new MediaTranscoder();
        var memoryStream = new InMemoryRandomAccessStream();
        var sampleBuffer = new Int16[1024];

        var ptResult = await transcoder.PrepareMediaStreamSourceTranscodeAsync(mediaStream, memoryStream, profile);

        while (more samples available ???)
        {
            Read samples somehow ???
            ProcessSamples(sampleBuffer, sampleCount):
        } }

    Can anyone help me complete the above code so I can process the PCM samples?

    Thanks in advance for any help.


    • Edited by BitFlipper Wednesday, June 18, 2014 4:51 PM
    Wednesday, June 18, 2014 3:57 PM

All replies

  • Oh, I didn't even know you could use MediaSource in WinRT apps!

    I implemented this using Media Foundation Source Reader. Here's the code:

    http://www.codeproject.com/Articles/459453/Code-to-stream-or-convert-MP-WMA-to-PCM-WAV-in-Wi

    Friday, June 20, 2014 3:41 AM
  • Lucian,

    Thanks for the link! I've actually been looking at that previously but seeing how much coding that would require (I'd need to convert to C# for one thing), I was hoping I could do it with the above sample code which seems very close to being complete for my purpose.

    Unfortunately my lack of knowledge regarding how to connect up MediaStreamSource, MediaEncoder etc is causing problems.

    Friday, June 20, 2014 11:51 AM
  • Hello,

    Have you seen this sample? What is missing from it that doesn't work for you?

    Transcoding media sample

    http://code.msdn.microsoft.com/windowsapps/Media-Transcode-Sample-f7ba5269

    Thanks,

    James

    Thursday, June 26, 2014 6:25 PM
  • James,

    I looked through the sample code but it doesn't show how to get to the actual audio PCM data. In order to process audio (apply DSP to it etc), you need access to the raw PCM data.

    Transcoding samples like this one don't help me because they just convert from one format to another and don't show how to get to the actual raw PCM data.

    Thanks

    Friday, June 27, 2014 12:58 AM
  • Check this sample

    http://code.msdn.microsoft.com/windowsapps/MediaStreamSource-media-dfd55dff/

    your point of interest is in Scenario1_LoadMP3FileForAudioStream.xaml.cs file

    at

    async void MSS_SampleRequested(Windows.Media.Core.MediaStreamSource sender, MediaStreamSourceSampleRequestedEventArgs args)

    MediaStreamSample sample = await MediaStreamSample.CreateFromStreamAsync(inputStream, sampleSize, timeOffset);
                    sample.Duration = sampleDuration;
                    sample.KeyFrame = true;

    it reading next sample, you can access sample raw buffer using Buffer property 

    Friday, June 27, 2014 12:08 PM
  • Hi Dmitry,

    I looked at that code but it is still not clear to me how the get the PCM data. As a test, I added the following after the code that you pointed out:

     

    var reader = DataReader.FromBuffer(sample.Buffer);
    var readSamples = sample.Buffer.Length / 4; // Format ?? PCM 16 ?, PCM 32 ?, PCM float ?
    var waveBuffer = new float[readSamples];
    
    for (var idx = 0; idx < readSamples; idx++)
    {
        waveBuffer[idx] = reader.ReadSingle();  // Format ??
    }

     

    However I tried all of the formats (Int16, Int32, float, double), but none of them appear to result in the actual PCM data.

    Any more help you can provide?

    Thanks, much appreciated.



    • Edited by BitFlipper Friday, June 27, 2014 1:29 PM
    Friday, June 27, 2014 1:28 PM
  • Good point, i forgot about format...  Currently i'm learning Win8 multimedia stack if i will find something more about format and MediaStreamSample i'll let you know.

    At a glance i see here only one solution - Windows Runtime Component to read samples using IMFSourceReader . This is more low-level approach, but in result you will get anything you need. 

    In my solution i'm passing IRandomAccessStream to MFCreateMFByteStreamOnStreamEx and creating reader using MFCreateSourceReaderFromByteStream .

    To get stream format and encoding properties i'm calling 

    m_spSourceReader->GetCurrentMediaType(streamIndex, spMediaType.GetAddressOf())


    On SampleRequest i'm reading samples from required stream using 

    ComPtr<IMFSample> spSample;

    hr = m_spSourceReader->ReadSample( streamIndex,// Stream index. 0,// Flags. &streamIndexActual,// Receives the actual stream index. &flags,// Receives status flags. &llTimeStamp,// Receives the time stamp. spSample.GetAddressOf()// Receives the sample or NULL. );

    IMFSample receives all the sample data i need, including buffers

    Here is a good post on how to use source reader http://msdn.microsoft.com/en-us/library/windows/desktop/dd389281(v=vs.85).aspx 

    Friday, June 27, 2014 3:19 PM
  • Ah thanks, I will look into that and report back whether it solves my problem.
    Friday, June 27, 2014 8:45 PM
  • Hello,

    The best solution for you is likely to use the Source Reader to grab the raw samples in the format that you need. There are other solutions as well such as writing an MFT plug-in for the Media Engine. But it really depends on what you plan to do with the uncompressed samples once you have them.

    Source Reader

    http://msdn.microsoft.com/en-us/library/windows/desktop/dd940436(v=vs.85).aspx

    I hope this helps,

    James


    Windows SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

    Thursday, July 03, 2014 6:06 PM
    Moderator
  • James,

    Thanks for the update. As to what I need the samples for... I need to perform DSP/analysis on the PCM data and display the results on the screen in realtime. For that I need the raw PCM samples.

    I did look at SourceReader (do you mean IMFSourceReader?) previously but was hoping that I can use existing .Net classes for my purpose. It seems the sample code I provided above is very close to giving me what I need.

    TBH, I'm not 100% sure what the relationships of these different sets of classes/interfaces are. It looks like IMFSourceReader is not directly available as a .Net class, but MediaStreamSource is. I remember reading that MediaStreamSource 

    The documentation is very sparse at this point so connecting all the dots is not straightforward.

    Thursday, July 03, 2014 6:51 PM