none
Binary download with WinHTTP

    Question

  •  

    Greetings to all. For a few days I've been coding a small app that uses WinHTTP, and this forum has been great help for common questios.
    Right now I got a working function that downloads a text file via WinHTTP. The problem is, that I need to do a binary download.
    I modified my text download code to see if it worked, but it doesn't.
    Im using a modified version of the sample code:

     

    Code Snippet

     DWORD dwSize = 0;
     DWORD dwDownloaded = 0;
     LPSTR pszOutBuffer;
     BOOL bResults = FALSE;
     HINTERNET hSession = NULL, 
     hConnect = NULL,
     hRequest = NULL;

     // Use WinHttpOpen to obtain a session handle.
     hSession = WinHttpOpen( L"WinHTTP Example/1.0", 
     WINHTTP_ACCESS_TYPE_DEFAULT_PROXY,
     WINHTTP_NO_PROXY_NAME, 
     WINHTTP_NO_PROXY_BYPASS, 0 );

     // Specify an HTTP server.
     if( hSession )
     hConnect = WinHttpConnect( hSession, L"www.microsoft.com",
     INTERNET_DEFAULT_HTTPS_PORT, 0 );

     // Create an HTTP request handle.
     if( hConnect )
     hRequest = WinHttpOpenRequest( hConnect, L"GET", NULL,
     NULL, WINHTTP_NO_REFERER, 
     WINHTTP_DEFAULT_ACCEPT_TYPES, 
     WINHTTP_FLAG_SECURE );

     // Send a request.
     if( hRequest )
     bResults = WinHttpSendRequest( hRequest,
     WINHTTP_NO_ADDITIONAL_HEADERS, 0,
     WINHTTP_NO_REQUEST_DATA, 0, 
     0, 0 );


     // End the request.
     if( bResults )
     bResults = WinHttpReceiveResponse( hRequest, NULL );

     // Keep checking for data until there is nothing left.
     if( bResults )
     {
     do 
     {
     // Check for available data.
     dwSize = 0;
     if( !WinHttpQueryDataAvailable( hRequest, &dwSize ) )
     printf( "Error %u in WinHttpQueryDataAvailable.\n",
     GetLastError( ) );

     // Allocate space for the buffer.
     pszOutBuffer = new char[dwSize+1];
     if( !pszOutBuffer )
     {
     printf( "Out of memory\n" );
     dwSize=0;
     }
     else
     {
     // Read the data.
     ZeroMemory( pszOutBuffer, dwSize+1 );

     if( !WinHttpReadData( hRequest, (LPVOID)pszOutBuffer, 
     dwSize, &dwDownloaded ) )
     printf( "Error %u in WinHttpReadData.\n", GetLastError( ) );
     else
     printf( "%s", pszOutBuffer );

     // Free the memory allocated to the buffer.
     delete [] pszOutBuffer;
     }
     } while( dwSize > 0 );
     }


     // Report any errors.
     if( !bResults )
     printf( "Error %d has occurred.\n", GetLastError( ) );

     // Close any open handles.
     if( hRequest ) WinHttpCloseHandle( hRequest );
     if( hConnect ) WinHttpCloseHandle( hConnect );
     if( hSession ) WinHttpCloseHandle( hSession );

     

     

    Any sugestions of how to make this code work with binary downloads? I'm thinking of some ofstream file with ios::binary but I couldn't make it work.

    Thanks in advance

    Saturday, May 03, 2008 6:37 AM

Answers

  • What kind of binary file?

     

    Your 6th parameter to WinHttpOpenRequest is WINHTTP_DEFAULT_ACCEPT_TYPES. If you read up on this, you will see that the implied mime type is "text/*", so only files of that type will be downloaded.

     

    Specify the mime type you are interested in, and make sure that the web server supports that mime type.

     

    Brian

     

    Monday, May 05, 2008 11:37 PM
  • It works now. I had to open the file in binary mode:

    fopen("file", w+b);

    Thus, one can download a resourece, keeping the MSDN sample code intact. The only additions being opening a file, writing to it using fwrite (in place of the printf to stdout), and  closing the file.

    Also note that  the url to the resource must be manually split into the host and path and each piece of this information passed to two different API calls.

    For example if you point to the rosource as follows:

    http://www.something.com/folder/file.xyz

    then make sure you pass www.something.com only to WinHttpConnect and /folder/file.xyz WinHttpOpenRequest.


    -------------------------

    And it won't work if you pass an ofstream object to a WinHTTP API. Use plane char buffers. And then 'write' the buffer to your streams. It'll work.






    Saturday, May 17, 2008 9:26 AM

All replies

  • What kind of binary file?

     

    Your 6th parameter to WinHttpOpenRequest is WINHTTP_DEFAULT_ACCEPT_TYPES. If you read up on this, you will see that the implied mime type is "text/*", so only files of that type will be downloaded.

     

    Specify the mime type you are interested in, and make sure that the web server supports that mime type.

     

    Brian

     

    Monday, May 05, 2008 11:37 PM
  • Hello

     

    Re: Binary download with WinHTTP

     

    I am going to mark this thread as answered since you have not followed up with any further information on your problem as requested - I assume you solved the problem yourself or one of the suggestions in this thread helped you solved the problem. If you have a solution you could post it so others can find it. If you don’t have a solution, then please submit further details and then mark the thread as unanswered.

     

    Thanks!

     

    Friday, May 09, 2008 8:56 AM
  • I think problem is in following statement,

     

    printf( "%s", pszOutBuffer );

     

    It will stop procedding further after first occurance of \0 i.e. a byte with numeric value 0.

     

    Instead directly write pszOutBuffer into file.

     

     

    Friday, May 09, 2008 5:56 PM
  • I have an identical problem. We can now move towards completing the discussion. I too am using the WinHttp sample from MSDN, and want to download an image (jpeg, gif, png...).

    Keeping the code unchanged ( i.e. passing WINHTTP_DEFAULT_ACCEPT_TYPES to WinHttpOpenRequest ) I am using fwrite to write to a file on disk. In the code I log the value of the dwDownloaded parameter passed to WinHttpReadData. I also log the bytes written by fwrite:

    Code Snippet

    fwrite(pszDataBuffer, (size_t)dwDownloaded, (size_t)1, fp);



    Both match, and they also match the size of the image residing in the web server, that I am downloading.

    But the downloaded file does not open in a picture viewer as an image. Also, the size of this downloaded file 'on disk' is almost double the actual size. How come? And why is the written data not a an image file?

    The above steps not having worked, I passed an array of zero-terminated strings to WinHttpOpenRequest, this way:


    Code Snippet

    TCHAR media_types[10][20];

    for(int i=0; i<10; i++)
    {
        ZeroMemory(media_types[i], 20);
    }
    StringCchCopy(media_types[0], 20, L"text/html");
    StringCchCopy(media_types[1], 20, L"image/pjpeg");
    StringCchCopy(media_types[2], 20, L"image/gif");

    WinHttpOpenRequest( hConnect, L"GET", resource_path,NULL,

    WINHTTP_NO_REFERER,                               (LPCWSTR*)media_types,                               WINHTTP_FLAG_BYPASS_PROXY_CACHE );


    //and the corresponding send request call is (unchanged):
    WinHttpSendRequest( hRequest,WINHTTP_NO_ADDITIONAL_HEADERS, 0,
                        WINHTTP_NO_REQUEST_DATA, 0,
                        0, 0 );


    This time WinHttpSendRequest fails with the following error:
    87: Parameter incorrect.

    Where's the mistake?

    Friday, May 16, 2008 9:10 AM
  • Well... If you write pszOutBuffer directly to the disk you get a corrupted file, because pszOutBuffer is a text buffer,not a binary buffer.
    Also If I do something like
    ofstram (myfile ios::binary | ios:Surpriseut)
    and then something like
    WinHttpReadData( hRequest, myfile, dwSize, &dwDownloaded )
    I also get a corrupted file.
    Any sugestions?

    Friday, May 16, 2008 3:02 PM
  • It works now. I had to open the file in binary mode:

    fopen("file", w+b);

    Thus, one can download a resourece, keeping the MSDN sample code intact. The only additions being opening a file, writing to it using fwrite (in place of the printf to stdout), and  closing the file.

    Also note that  the url to the resource must be manually split into the host and path and each piece of this information passed to two different API calls.

    For example if you point to the rosource as follows:

    http://www.something.com/folder/file.xyz

    then make sure you pass www.something.com only to WinHttpConnect and /folder/file.xyz WinHttpOpenRequest.


    -------------------------

    And it won't work if you pass an ofstream object to a WinHTTP API. Use plane char buffers. And then 'write' the buffer to your streams. It'll work.






    Saturday, May 17, 2008 9:26 AM
  • Thanks a lot, that worked pretty fine. Smile
    Monday, May 19, 2008 7:08 PM
  • For example if you point to the rosource as follows:

    http://www.something.com/folder/file.xyz

    then make sure you pass www.something.com only to WinHttpConnect and /folder/file.xyz WinHttpOpenRequest.

    Where do i specify /folder/file.xyz in WinHttpOpenRequest. I mean to ask which parameter would take that.
    Tuesday, May 12, 2009 8:48 AM
  • Well I have been finally been able to the above task. Somehow all the above codes were not running for me. So for all the poor souls like me, i would give them the code which worked for me...
    #include <winsock2.h>
    #include <Winhttp.h>
    //#include <urlmon.h>
    #include <windows.h>
    #include <iostream>
    #include <fstream>
    using namespace std;
    
    int main (void)
    {
    	//Variables 
    	DWORD dwSize = 0;
    	DWORD dwDownloaded = 0;
    	LPBYTE pszOutBuffer;
    	
    	BOOL  bResults = FALSE;
    	HINTERNET  hSession = NULL, hConnect = NULL, hRequest = NULL;
    	// Use WinHttpOpen to obtain a session handle.
    	hSession = WinHttpOpen( L"WinHTTP Example/1.0", WINHTTP_ACCESS_TYPE_DEFAULT_PROXY, WINHTTP_NO_PROXY_NAME, WINHTTP_NO_PROXY_BYPASS, 0);
    	// Specify an HTTP server.
    	if (hSession)    
    		hConnect = WinHttpConnect( hSession, L"192.168.20.69", INTERNET_DEFAULT_HTTP_PORT, 0);
    	// Create an HTTP request handle.
    	if (hConnect)    
    		hRequest = WinHttpOpenRequest( hConnect, L"GET", L"/xyz/1.txt", NULL, WINHTTP_NO_REFERER, NULL, NULL);
    	// Send a request.
    	if (hRequest)    
    		bResults = WinHttpSendRequest( hRequest, WINHTTP_NO_ADDITIONAL_HEADERS, 0, WINHTTP_NO_REQUEST_DATA, 0, 0, 0);
    	// End the request.
    	if (bResults)    
    		bResults = WinHttpReceiveResponse( hRequest, NULL );/**/
    	// Keep checking for data until there is nothing left.
    	HANDLE hFile = CreateFile("C:\\test1.txt", GENERIC_READ|GENERIC_WRITE, FILE_SHARE_READ, NULL, OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL); 
    	if (bResults)    
    		do     
    		{        
    			// Check for available data.        
    			dwSize = 0;        
    			if (!WinHttpQueryDataAvailable( hRequest, &dwSize))
    				printf( "Error %u in WinHttpQueryDataAvailable.\n", GetLastError());        
    			// Allocate space for the buffer.        
    			pszOutBuffer = new byte[dwSize+1];        
    			if (!pszOutBuffer)        
    			{            
    				printf("Out of memory\n");            
    				dwSize=0;        
    			}        
    			else        
    			{            
    				// Read the Data.            
    				ZeroMemory(pszOutBuffer, dwSize+1);            
    				if (!WinHttpReadData( hRequest, (LPVOID)pszOutBuffer, dwSize, &dwDownloaded))                
    				{                
    					printf( "Error %u in WinHttpReadData.\n", GetLastError());                
    				}            
    				else                
    				{                        
    					//printf("%s", pszOutBuffer); 
    					DWORD wmWritten;
    					bool fr = WriteFile(hFile, pszOutBuffer, dwSize, &wmWritten, NULL);
    					int n = GetLastError();              
    				}            
    				// Free the memory allocated to the buffer.            
    				delete [] pszOutBuffer;        
    			}    
    		} while (dwSize>0);
    		CloseHandle(hFile);
    		// Report any errors.
    		if (!bResults)    
    			printf("Error %d has occurred.\n",GetLastError());
    		// Close any open handles.
    		if (hRequest) WinHttpCloseHandle(hRequest);
    		if (hConnect) WinHttpCloseHandle(hConnect);
    		if (hSession) WinHttpCloseHandle(hSession);
    }
    


    In here "1.txt" is the file and my Virtual directory alias is "xyz"
    Am saving this file in C:/ Drive.

    Thanks a lot.
    Harsh
    • Proposed as answer by harshkumar1 Tuesday, May 19, 2009 7:59 AM
    Tuesday, May 19, 2009 7:59 AM