locked
WASAPI Initialized with success, only capturing static RRS feed

  • Question

  • I'm trying to test WASAPI recording functionality, and I'm running into a bit of trouble. Previously, I was able to use WASAPI on WP8 to capture and render audio. I've took a similar approach to the audio hardware/software setup for the desktop, but right now I'm only getting static when capturing.

    First, I use the IMMDevice API to enumerate the devices and either get the default device or one of my choosing. All function calls return successfully, and I'm able to print the information (friendly name, ID, etc) of the device chosen. Besides this part, device discovery, my Desktop implementation mimics very closely my working WP8 implementation.

    Next, I Activate() an IAudioClient interface. I'm able to Initialize() this interface with both the default Mix Format (Stereo, 32 bit, 48000) and one with modifications (Mono, 16 bit, 48000, WAVE_FORMAT_PCM). The interface is Initialize()'d with the StreamFlags 0x88140000--

    (This was my first point of concern. This worked for my WP8 app, as the stream flags were suggested by the Chatterbox VoIP sample. This post indicates that high bits of the first two hex values are used internally and are not documented publicly. When I use StreamFlags of 0x00040000, I have to use the default Mix Format. However, my problem persists with either Initialization)

    -- I can then create the capture event handle and bind it to the interface with SetEventHandle(). Finally, I GetService() to retrieve the IAudioCaptureClient interface, and I Start() the device successfully. 

    Nothing in this setup chain fails (All return S_OK).

    I then kick off the processing thread, which Waits (WaitForSingleObjectEx) on the capture event handle. This keeps the loop on a rough 10ms (confirmed with QueryPerformanceCounter). I GetBuffer(), which grabs 480 frames each call, write the bytes to a file, then ReleaseBuffer().

    I'm testing on a laptop, using the built in microphone. The device works, and I've verified it can record audio. However, I only see heavy static noise when I capture on that device using my sample program. Even if the Initialization phase appears successful, could there be something causing the microphone to not record properly? I'll upload some real code in a bit.

    Thanks in advance for the help.

    ______

    UPDATE:

    I'm using Audacity to test my sample recordings and to test the onboard audio hardware of my laptop. When the "Audio Host" is either MME or DirectSound, my laptop's internal mic is an input option. However, when the Audio Host is WASAPI, the internal mic is not present.  What's going on here? Can WASAPI not handle certain devices? If so, is there a way to distinguish between supported and non supported devices?


    • Edited by Jim M B Wednesday, June 25, 2014 11:38 PM
    Wednesday, June 25, 2014 9:46 PM

All replies

  • HRESULT hr = S_OK;
    
    	// Get capture endpoint from enumator
    	if (NULL == pCaptureDevice)
    		hr = pCaptureEnumerator->GetDefaultAudioEndpoint(eCapture, eCommunications, &pCaptureDevice);
    
    	if (SUCCEEDED(hr)) // Activate the capture interface on the default endpoint
    	{
    		printf("CAPTURE INTERFACE\n\n");
    		displayDeviceInfo(pCaptureDevice);
    		hr = pCaptureDevice->Activate(IID_IAudioClient, CLSCTX_ALL, NULL, (void**)&pCaptureClient);
    
    		if (SUCCEEDED(hr)) // Get Audio format
    		{
    			WAVEFORMATEX *pwfx = NULL;
    			hr = pCaptureClient->GetMixFormat(&pwfx);
    
    			if (NULL == pwfx)
    			{
    				hr = HRESULT_FROM_WIN32(GetLastError());
    			}
    
    			if (SUCCEEDED(hr)) // Set Audio format and Init device
    			{			
    				if ( SUCCEEDED(tryFormat(pCaptureClient, pwfx, 1, 16)) )
    					printf("\nCustom waveform accepted:");
    				else
    					printf("\nCompromise waveform:");
    
    				printf("\n\tnChannels (%d) \n\tnSamplesPerSec (%d) \n\tBitsPerSamples (%d)\n", pwfx->nChannels, pwfx->nSamplesPerSec, pwfx->wBitsPerSample);
    
    				DWORD flags = 0;
    				flags |= AUDCLNT_STREAMFLAGS_EVENTCALLBACK | AUDCLNT_STREAMFLAGS_CROSSPROCESS;
    
    				hr = pCaptureClient->Initialize(
    					AUDCLNT_SHAREMODE_SHARED,			// Share mode
    					AUDCLNT_STREAMFLAGS_EVENTCALLBACK,							// Stream flags (previously 0x88140000)
    					100 * 10000,						// Buffer duration in hNS, (x10000 = ms) 100ms
    					0,									// Periodicity (must be 0 for non-exclusive mode, aka shared)
    					pwfx,								// Audio format
    					NULL);								// Session GUID
    
    				if (SUCCEEDED(hr)) // Register mic capture event
    				{
    					hCaptureEvent = CreateEventEx(NULL, NULL, 0, EVENT_ALL_ACCESS);
    					if (NULL == hCaptureEvent)
    					{
    						hr = HRESULT_FROM_WIN32(GetLastError());
    					}
    					if (SUCCEEDED(hr))
    					{
    						hr = pCaptureClient->SetEventHandle(hCaptureEvent);
    
    						if (SUCCEEDED(hr)) // Get access to capture stream 
    						{
    							hr = pCaptureClient->GetService(IID_IAudioCaptureClient, (void**)&pCaptureStream);
    						}
    						else
    							printf("Error: hCaptureEvent not bound to mic event");
    					}
    					else
    						printf("Error: hCaptureEvent not created correctly");
    				}
    				else
    					printf("Error: Failed to initialize device\n");
    			}
    			else
    				printf("Error: Failed to GetMixFormat\n");
    		}
    		else
    			printf("Error: Failed to Activate m_pDefaultCaptureDevice\n");
    	}
    	else
    	{
    		printf("Error: Failed to GetDefaultAudioEndpoint");
    
    		if (E_NOTFOUND == hr)
    			printf("\nt\tNo default capture device (microphone) was detected on this system");
    	}

    This is the Microphone setup/init
    • Edited by Jim M B Wednesday, June 25, 2014 9:50 PM
    Wednesday, June 25, 2014 9:49 PM
  • 	while (SUCCEEDED(hr))
    	{
                    // Some timing stuff
    
    		/* Capture */
    		WaitForSingleObjectEx(hCaptureEvent, INFINITE, FALSE);
    		hr = pCaptureStream->GetBuffer(&pbData, &nFrames, &dwFlags, nullptr, nullptr);
    
    		if (SUCCEEDED(hr))
    		{
    			/* Resampling will go here*/
    
    			// Write to file to test
    			fwrite(pbData, sizeof(SHORT), nFrames, file);
    
    			/* Rendering will go here */
    
    			/* Control will go here */
    			
    		}
    
    		hr = pCaptureStream->ReleaseBuffer(nFrames);
            }
    Processing loop (only capture is implemented for now). Thread is created after devices are Start()ed
    • Edited by Jim M B Wednesday, June 25, 2014 9:53 PM
    Wednesday, June 25, 2014 9:53 PM
  • There's a Windows sample that shows how to use WASAPI here. Can it capture from your internal microphone? What does the audio quality sound like?

    http://code.msdn.microsoft.com/windowsapps/Windows-Audio-Session-22dcab6b

    Is there a reason you want to use IMMDeviceEnumerator rather than Windows.Media.MediaDevice.GetAudioCaptureSelector and .GetDefaultAudioCaptureId?

    > 0x88140000

    Don't pass undocumented stream flags. The sample is bogus.

    > fwrite(pbData, sizeof(SHORT), nFrames, file);

    This seems very fragile; first, it assumes a mono 16-bit integer format. (A potential fix would be to pass pwfx->nBlockAlign instead of sizeof(SHORT).) Second, if your fwrite(...) call gets held off, you'll glitch; it is more performant to stick the buffer somewhere and then lazily write it to disk off of the capture thread.


    Matthew van Eerde

    Thursday, June 26, 2014 12:12 AM
  • Thanks for the response. I've downloaded the sample and will be trying it out shortly.

    It had been awhile since I touched WASAPI, so I was reading on the Core Audio APIs, where I saw the MMDevice API. It seemed straightforward. I assume using MediaDevice is better?

    So, without passing the undocumented stream flags, it seems like I can't change anything in the Mix Format. Do I just need to suck it up and resample to get the formatting I want? I'm not too keen on capturing in Stereo, 48k, 32 bit depth. It seems excessive.

    I'll fix the fwrite() immediately. I was throwing things together hastily just to get compiled and running.

    EDIT:

    Does the sample project require 8.1? I only have access to machines with 8 on them. I tried downloading and openening the project multiple times, but VS2013 says its Unable to read the project file "WASAPIAudio.vcxproj"

    • Edited by Jim M B Thursday, June 26, 2014 12:43 AM
    Thursday, June 26, 2014 12:32 AM
  • If you want to capture in a particular format, WASAPI is not for you; try a higher-level API like MediaCapture. There's a sample here: http://code.msdn.microsoft.com/windowsapps/media-capture-sample-adf87622

    Yes, the version I gave you requires Windows 8.1 and Visual Studio 2013. If you want to run on Windows 8.0, you can download the Windows 8.0 sample pack: http://code.msdn.microsoft.com/windowsapps/Windows-8-app-samples-4d76cbbf

    Updating from Windows 8.0 to Windows 8.1 is free.


    Matthew van Eerde

    Thursday, June 26, 2014 8:52 PM
  • My only necessity is capturing PCM; I can handle the resampling if that's all it takes. I was able to do just that in my phone app. I'm just hung up on the "successful initialization" leading to capturing static.

    I don't have the privileges to upgrade the machines I'm working on to 8.1, but I'm in talks with the IT dept now. I'll look at those W8 samples for the time being.

    Thursday, June 26, 2014 10:08 PM
  • I have Windows 8.1 running on my machine now. I didn't realize that the first sample you linked to was for Windows Store apps instead of Windows Desktop. Trying to sort out some VS stuff to finally get a sample running.

    I'm slowly figuring some things out about VS. I have VS2013 for Windows Desktop, which I intend to use to develop my app. So in order to get a WASAPI sample working on my system, I need to set up a whole new VS environment: Either 1) the previous version, VS2012, or 2) one intended for Metro apps, VS2013 for Windows

    EDIT:

    I got the sample running on VS2013 for Windows. The sample app is able to capture audio just fine. Now trying to model my code after this sample.

    And as for using the Windows.Media.Devices namespace, I can't. The namespace has a minimum client of W8, and I need this app to run on W7 as well.


    • Edited by Jim M B Monday, June 30, 2014 11:07 PM
    Monday, June 30, 2014 9:46 PM
  • Still playing around with this issue. 

    My latest approach was to copy This Sample. The only changes I make to the setup is changing the WaveFormat's tag to WAVE_FORMAT_PCM, setting the bit depth to 16, and setting cbSize to 0. The Initialize() call returns success, and the rest of the code is untouched. I copy each sample into a large local buffer, then write that buffer to file after the capturing is done. 

    Upon closer inspection of the captured audio, I'm seeing irregular but frequent spurts of 0's in the data. AUDCLNT_BUFFERFLAGS_SILENT is not returned from the GetBuffer() call, nor are any flags for that matter. Here is a screencap of a small segment of the recorded audio, viewed in Audacity. You can see a few segments of 0's, some lasting longer than 10ms:

    Thursday, July 10, 2014 5:54 PM
  • Are you getting the same behavior from (say) the built-in SoundRecorder app?

    If the SoundRecorder app works, are you registering your streaming thread with the Multimedia Class Scheduler service, either by using Media Foundation work queues as the WASAPI sample does, or by calling AvSetMmThreadCharacteristics directly?

    If you're not registered with MMCSS, and your thread gets held off, that would explain the packets of silence.


    Matthew van Eerde

    Thursday, July 10, 2014 9:20 PM
  • The SoundRecorder app works just fine. 


    I had SetThreadPriority() to THREAD_PRIORITY_TIME_CRITICAL -- is this not sufficient to keep my processing thread from being bumped?  I'll try registering the thread with the MMCSS now.

    EDIT:

    I've tried using AvSetMmThreadCharacteristics() as per the usage in this example. I'm still seeing broken audio.


    • Edited by Jim M B Friday, July 11, 2014 4:33 PM
    Thursday, July 10, 2014 10:11 PM
  • OK, so you're registering with MMCSS, but you're still seeing glitches.

    The Media eXperience Analyzer tool ( http://www.microsoft.com/en-us/download/details.aspx?id=43105&WT.mc_id=rss_windows_allproducts ) is a useful tool for troubleshooting glitches; you might give that a try.

    I would take a very close eye at everything you do in your processing pass and ask the question "could this block".

    Another technique is to call QueryPerformanceCounter() just after you're woken up and just before you call WaitForSingleObject again. If there's a large delta between the two (more than about 5 ms or so), the problem is on your end: something you're doing is blocking. If there's a large delta between waiting and being woken up (more than 10 ms) then something is holding off the audio engine.

    Or it could be something less complicated, like you're being given the right data at the right time, but you're not writing it out correctly.


    Matthew van Eerde

    Friday, July 11, 2014 7:23 PM
  • For the sake of simplicity, I'm just trying to get this sample code to work, which doesn't use the event driven capture mode. There is no external code, no additional threads. Just this sample code in its own project, with the set up and capture of a few seconds of audio (And, of course, elevating the threads priority).

    On each iteration of the loop, the sample waits for half of the microphone buffer's duration (500ms), then proceeds to read as much data as possible from the IAudioCaptureClient interface using GetBuffer().

    I have a global buffer, declared to have enough space for 5 seconds of 32-bit 44.1KHz data (I only capture 3 seconds of 16-bit 44.1KHz, so this buffer should be more than big enough). To write in this buffer, I use a tracking pointer that gets incremented after each write.

    BYTE wBuf[44100 * 4 * 5];

    BYTE* pBuf = wBuf;

    So, in the processing loop of the sample, we capture:

    hr = pCaptureClient->GetBuffer(
        &pData,
        &numFramesAvailable,
        &flags, NULL, NULL);
    and write to the audio sink. In place of the stub
    hr = pMySink->CopyData(pData, numFramesAvailable, &bDone);

    I have implemented

    if (pData)
    {
        int bytesToWrite = (numFramesAvailable * pwfx->nBlockAlign);
        memcpy(pBuf, pData, bytesToWrite);
        pBuf += bytesToWrite;
        totalBytes += bytesToWrite;
    }

    Once I've accumulated all the data, and tracked the total number of bytes put in the local buffer, I write the buffer to file by calling:

    UINT error = fopen_s(&fp, "testaudio", "w+");

    int numBytesWrittenToFile = fwrite(wBuf, 1, totalBytes, fp);

    fclose(fp);

    Does the copy/write logic seem sound to you? Let's confirm/eliminate the simple explanation while I look into the Analyzer tool. I appreciate the help.


    • Edited by Jim M B Tuesday, July 15, 2014 9:15 PM
    Tuesday, July 15, 2014 9:08 PM
  • Sure, that looks reasonable; it should be fine to copy into your locally allocated buffer between GetBuffer() and ReleaseBuffer().

    It would be interesting to log timestamps before and after the Sleep().


    Matthew van Eerde

    Wednesday, July 16, 2014 8:49 PM
  • In the example, with a second-long microphone buffer, timing with QueryPerformanceCounter indicates that the program is sleeping for 500ms (which it should be). In my first attempt, timing with QueryPerformanceCounter indicates the processing loop runs ~10ms (sometimes 9ms, but that should be fine).

    I'm a little confused on how to properly analyze an ETW file using the MXA, but I started looking into the event logs detailed by this article. I found the event providers linked to my program, and started looking into them.

    Looking in Computer Management -> System Tools -> Event Viewer -> Apps and Services Logs -> Microsoft -> Windows -> Audio, the GlitchDetection log notes a bunch of events when I run my program. There are two events, ID 35 and 33 respectively. Here are two samples:

    Engine Glitch: CP Server Output Endpoint - Read Pointer
      pCCrossProcessServerOutputEndpoint=[0x5f78960070]
      WriteOffset=[2048]
      ReadOffset=[2048]
      BytesToWrite=[882]
    
    Engine Glitch: CP Server Input Endpoint - Starvation:
      pCCrossProcessServerInputEndpoint=[0x5f78960770]
      WriteOffset=[2048]
      ReadOffset=[2048]
      BufferSize=[17640]
      BytesAvail=[1764]

    I'm trying to find more documentation or information about these glitches, but I'm struggling a little right now.



    • Edited by Jim M B Wednesday, July 16, 2014 9:28 PM
    Wednesday, July 16, 2014 9:13 PM
  • Those events mean that the audio engine went to write data (that it got from the microphone) into the cross-process buffer, but the buffer was already full.

    The trace seems to indicate that the buffer size is 17640 bytes, which is enough to hold 100 ms of 44.1 stereo int16 data. That's plenty of data; it should be possible to run glitch-free with a buffer size as small as 20 ms. You just need to wake up every 10 ms and drain the buffer so the audio engine has room to write.

    But you're only waking up every 500 ms, which changes the game. Instead of using Sleep(...), use event-driven mode and have the engine wake you up when it has data ready for you.

    Or if you really want to wake up every 500 ms, ask IAudioClient::Initialize for a larger buffer; say, a 1000 ms buffer.


    Matthew van Eerde

    Thursday, July 17, 2014 12:00 AM
  • Perhaps I should have been a bit more clear: Those events were triggered from the "Capturing a Stream" example code, which doesn't use the event driven mode. I'm trying both event driven (code at the top of this post) and non event driven (sample app) capture, both with the same results.

    Sp, in the sample, The Initialize() call in the sample code does actually allocate a buffer of 1000ms. It then wakes up every 500ms and calls GetBuffer() many times consecutively.

    __

    In my app, which DOES implement event driven capture (as per the code at the top of this post), I still receive the same glitch events:

    Engine Glitch: CP Server Output Endpoint - Read Pointer Overwrite: 
      pCCrossProcessServerOutputEndpoint=[0x7664e40070]
      WriteOffset=[2930]
      ReadOffset=[2930]
      BytesToWrite=[882]
    
    Engine Glitch: CP Server Input Endpoint - Starvation: 
      pCCrossProcessServerInputEndpoint=[0x7664e403f0]
      WriteOffset=[2048]
      ReadOffset=[2048]
      BufferSize=[17640]
      BytesAvail=[1764]

    The mic buffer is Initialize()d to hold 100ms of data

    QueryPerformanceCounter() calls indicate my processing loop runs every ~10ms

    I grab 441, 16-bit samples every time


    • Edited by Jim M B Thursday, July 17, 2014 12:54 AM
    Thursday, July 17, 2014 12:47 AM
  • > QueryPerformanceCounter() calls indicate my processing loop runs every ~10ms

    This is expected; that is, you should expect to be woken up once per device period, which is 10 ms or slightly above.

    How long does your processing take on each pass - that is, how much time elapses between when you are woken up and when you go to sleep again?

    > I grab 441, 16-bit samples every time

    Don't do that; drain the buffer completely every time you're woken up. More specifically, call IAudioCaptureClient::GetNextPacketSize() until you're told the next packet size is zero. If all is well, this will result in your grabbing 441 samples every time; but if you miss a beat, you'll get 882 samples on the next pass.


    Matthew van Eerde

    Thursday, July 17, 2014 3:10 AM
  • Now draining the buffer every time. Still getting the glitch events and broken audio.
    Thursday, July 17, 2014 4:41 PM
  • Can you send me your code? mateer at microsoft dot com

    Matthew van Eerde

    Thursday, July 17, 2014 5:46 PM
  • Should be on its way now.
    Thursday, July 17, 2014 6:50 PM
  • Looks like fwrite() was the culprit. I guess either pbData was being misinterpreted, or some newlines were being added. Either way, the PCM data was being garbled. I used Createfile() and WriteFile() instead to get working audio.
    • Edited by Jim M B Monday, August 25, 2014 9:15 PM
    Monday, August 25, 2014 9:14 PM