WinHttp advice

Question

Hi all.

First of all I appreciate very much how are you doing for having this nice library. Because we wanted to use something in production and at that time it seemed this library was not ready for production use we implemented our own WinHttp wrapper classes to use on a server.

I have some questions that I hope you can address considering you implemented this library on top of WinHttp :-)

1. On very high demand (where I have to open/serve maybe thousands of connections - which I obviously cannot handle if they keep coming at a very high rate) I set myself a limit on a number of outstanding requests at a given moment and don't start any new connections in that case. Event if I set is quite low I see that my process keep accumulating memory because of calls into WinHttp operator new(new connection created with WinHttpConnect).

At some point if I keep pumping requests and don't wait for memory consumed bu winhttp module to free up it crashes (at stress testing)

Function   Source   Destination 
winhttp!operator new+16      ntdll!RtlAllocateHeap 
winhttp!RMakeInternetConnectObjectHandle+12      winhttp!operator new 
winhttp!WinHttpConnectInternal+7a      winhttp!RMakeInternetConnectObjectHandle 
winhttp!WinHttpConnect+a4      winhttp!WinHttpConnectInternal 
WebMon_Categorization!WinHttp::Connection::Initialize+4c   d:\dev\webmonitor\dev\2012 sr2\src\webmon.categorization\http.h @ 113 + 19   winhttp!WinHttpConnect 
WebMon_Categorization!WinHttp::Client<WebMonCategorization::CategorizationAsyncRequestHandler>::Client<WebMonCategorization::CategorizationAsyncRequestHandler>+9a   d:\dev\webmonitor\dev\2012 sr2\src\webmon.categorization\http.h @ 162   WebMon_Categorization!WinHttp::Connection::Initialize 
WebMon_Categorization!WebMonCategorization::BC::StartLookup+239 
 
 

2. What is the best way to deal with this ? My implementation does not use tasks in order to process efficiently the responses itself but as I've told you I have memory issues because of the requests itself (even though I imposed myself a limit of total outstanding requests at a moment of 50 for example, memory keeps growing slowly but steadily).  I mention that my process handle count does not increase so it seems that I don't suffer from a handle leak (first thing that I've checked)

Answer 1

I've found out the reason I was leaking memory. It was because I wasn't using winhttp async API correctly. If anyone is interested I will copy here the status callback handling that seems to be working as of now.

As for processing the data received from IO I made the processing outside of winhttp callbacks to utilize better CPU resources (using VS10 ppl in my case)

Answer 2

Hy. It seems that I still get memory leaks reported in winhttp.dll, because of the way I'm using winhttp API as it seems. Can anyone tell me if there is something wrong in the way I'm handling requests status calbacks down here ?

template <typename T>
void WinHttp::AsyncRequest<T>::OnCallback(DWORD code, const void* info, DWORD length)
{
	T* pT = static_cast<T*>(this);

	switch (code)
	{
	case WINHTTP_CALLBACK_STATUS_SENDREQUEST_COMPLETE:
	case WINHTTP_CALLBACK_STATUS_WRITE_COMPLETE:
		{
			HRESULT result = pT->OnWriteData();
			if (FAILED(result))
			{
				throw CommunicationException(::GetLastError());
			}
			if (S_FALSE == result)
			{
				if (!::WinHttpReceiveResponse(handle, 0)) // reserved
				{
					throw CommunicationException(::GetLastError());
				}
			}
			break;
		}

	case WINHTTP_CALLBACK_STATUS_HEADERS_AVAILABLE:
		{
			DWORD statusCode;
			DWORD statusCodeSize = sizeof(DWORD);
			if (!::WinHttpQueryHeaders(handle, WINHTTP_QUERY_STATUS_CODE | WINHTTP_QUERY_FLAG_NUMBER, WINHTTP_HEADER_NAME_BY_INDEX,	&statusCode, &statusCodeSize, WINHTTP_NO_HEADER_INDEX))
			{
				throw CommunicationException(::GetLastError());
			}
			pT->OnStatusCodeReceived(statusCode);
			if (!::WinHttpQueryDataAvailable(handle, 0))
			{
				// If a synchronous error occured, throw error.  Otherwise
				// the query is successful or asynchronous.
				throw CommunicationException(::GetLastError());
			}

			break;
		}

	case WINHTTP_CALLBACK_STATUS_DATA_AVAILABLE:
		{
			unsigned int size = *((LPDWORD) info);
			if (size == 0)
			{
				pT->OnResponseComplete(S_OK);
			}
			else
			{
				unsigned int sizeToRead = (size <= chunkSize) ? size : chunkSize;
				if (!::WinHttpReadData(handle, &buffer[0], sizeToRead, 0)) // async result
				{
					throw CommunicationException(::GetLastError());
				}
			}
			break;
		}

	case WINHTTP_CALLBACK_STATUS_READ_COMPLETE:
		{
			if (length != 0)
			{
				pT->OnReadComplete(&buffer[0], length);
				if (!::WinHttpQueryDataAvailable(handle, 0))
				{
					// If a synchronous error occured, throw error.  Otherwise
					// the query is successful or asynchronous.
					throw CommunicationException(::GetLastError());
				}
			}
			else
			{
				pT->OnResponseComplete(S_OK);
			}

			break;
		}

	case WINHTTP_CALLBACK_STATUS_REQUEST_ERROR:
		{
			{
				throw CommunicationException(::GetLastError());
			}

			break;
		}
	}
}

Here buffer is a vector<char> that has reserved 8K once the request is initiated. Thanks in advance.

In OnResponseComplete, OnResponsEerror I eventually call also:

				::WinHttpSetStatusCallback(handle, NULL, 0, 0);
				assert(::WinHttpCloseHandle(this->handle));
				this->handle = nullptr;

-Ghita

Answer 3

Anyone, any idea to try that out ? I don't seems to find what is making memory to leak (allocated by winhttp.dll).

Answer 4

Sorry, Ghita, nothing springs to mind immediately when I look at your code. This forum is probably not the one where you get the broad exposure to your question that I suspect that you need. These asynchronous APIs can be tricky to use sometimes (one of the things we're trying to address with Casablanca).

I suggest re-posting your question on some forum that is more directly dedicated to WinHttp, where you probably will get more people looking at the problem with you.

Thanks,

Niklas

Answer 5

Thanks for suggestion Niklas, I've already posted this on stackoverflow but there also limited results because it's too specialized...

One question though if anyone knows. Is it safe to call 

::WinHttpSetStatusCallback(handle, NULL, 0, 0);
assert(::WinHttpCloseHandle(this->handle));

from the context of the request status callback ? What I do is practically close the request (and then the corresponding connection handle afterwards) from the context of the callbacks: when I get STATUS_RED_COMPLETE (with length = 0), STATUS_REQUEST_ERROR, STATUS_DATA_AVAILABLE(with size = 0) or when any of the WinHttp API called inside the status calback return FALSE (as seen in code above).

Answer 6

Found out some hints on how to properly handle, especially closing of connections  from a MS employee (I queses he is still is :-)), I will not mention his name here as I've didn't discuss about that with him. I will keep you posted as I'll be able to tackle this problem fully.

Answer 7

Sorry, I don't know.

Niklas

Answer 8

Thanks for your support Niklas. Found out details about how to fix my issues from Kenny Kerr. More details are here:

http://stackoverflow.com/questions/11577503/winhttp-used-in-async-mode-error-internet-cannot-connect-how-to-cleanly-close