Processing audio from the microphone in realtime

Question

I want to process audio in realtime from the microphone, but the documentation is so incomplete that it is impossible to figure out how to do it. I believe I need to use MediaCapture.PrepareLogLagRecordToStreamAsync. Note this needs to work on both Windows 8.1 and WP 8.1.

Unfortunately the only samples I found simply saves the recording to a file, however I need to be able to stream the data into memory so that I can process it in realtime. I tried using InMemoryRandomAccessStream and can retrieve the first second or so of data after which it stops.

This is what I'm doing right now:

var capture = new MediaCapture();
var stream = new InMemoryRandomAccessStream();
var settings = new MediaCaptureInitializationSettings();
var profile = MediaEncodingProfile.CreateWav(AudioEncodingQuality.Auto);

settings.StreamingCaptureMode = StreamingCaptureMode.Audio;
await capture.InitializeAsync(settings);

var recording = await capture.PrepareLowLagRecordToStreamAsync(profile, stream).AsTask();

await recording.StartAsync().AsTask();

Can someone show me how to then continuously retrieve the data from the stream? It only works for a second or so after which new data stops being recorded into the stream. Do I need to call PrepareLowLagRecordToStreamAsync after each time I read the data out of the stream?

Answer 1

OK I looked at the actual data I'm getting in the stream. It appears this is not raw audio data but possibly a WAV file contents with the RIFF file header etc (or maybe even an image as this is what the documentation hints at). This is obviously not what I want.

So how does one capture raw audio from the microphone?

Answer 2

Windows Audio Session API (WASAPI)

See http://code.msdn.microsoft.com/windowsapps/Windows-Audio-Session-22dcab6b

Answer 3

Windows Audio Session API (WASAPI)

See http://code.msdn.microsoft.com/windowsapps/Windows-Audio-Session-22dcab6b

These don't appear to be available for Windows Phone 8.1. Or if they are, probably only for backwards compatibility. What would be the right APIs to use for a Universal App that targets both Windows 8.1 and Windows Phone 8.1 without using obsoleted APIs?

Answer 4

Hello,

Rob is correct. WASAPI is our low level audio API on all platforms including WP8 and WP8.1. If you need low access to individual audio packets this is an option.

If you want to use the MediaCapture element. You can write a plug-in and run your processing code in the plug-in in real time.

Unfortunately WASAPI and media plug-ins are only supported from C++ CX.

MediaCapture class

http://msdn.microsoft.com/en-us/library/windowsphone/develop/windows.media.capture.mediacapture.aspx

MediaCapture.AddEffectAsync

http://msdn.microsoft.com/en-us/library/windowsphone/develop/br211961.aspx

I hope this helps,

James

---

Windows SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

Answer 5

James, thanks for confirming. The documentation seems to be very sparse at this point.

Answer 6

Have you seen the WASAPI sample?

http://code.msdn.microsoft.com/windowsapps/Windows-Audio-Session-22dcab6b

-James

---

Windows SDK Technologies - Microsoft Developer Services - http://blogs.msdn.com/mediasdkstuff/

Answer 7

James,

Yes I did, thanks. I have it working now in a C# project.

Answer 8

Hello BitFlipper,

I am trying to do an Universal App in order to process the audio from the microphone in realtime. I am reading your posts and I finally used the wrapper you suggested on the channel9 thread.

Could you send me some sample code, since actually you seem the only one that resolved this big problem on Universal App.

Thanks in advance