locked
Need help on partial httpwebrequest RRS feed

  • Question

  • User-1702225963 posted

    Hi,

      I've been trying to download a file as parts using multiple threads in C#.  My question is how can i request the thread to donwload only a range of bytes by specify starting byte and ending byte of a file whose location and size are know.

    My code below doesn't work.

    webRequest = (HttpWebRequest)WebRequest.Create(SOURCE_FILENAME);

    webRequest.ServicePoint.ConnectionLimit = 500;

    webRequest.AddRange(
    Convert.ToInt32(StartPoint),Convert.ToInt32(EndPoint));

    webRequest.Credentials = CredentialCache.DefaultCredentials;

    webResponse = (HttpWebResponse)webRequest.GetResponse();

     

    return webResponse.GetResponseStream();

     

    Thanks in advance,

     

    Tuesday, April 1, 2008 4:41 PM

Answers

  • User242249085 posted

    The HttpWebRequest.AddRange(startbyte,endbyte) doesn't work.  The server returns the full range of stream.
     

    Not all servers support range requests, you need to handle the case where the server does not.

    I would also strongly suggest using the IO thread pool  by using request.BeginGetResponse() rather than spinning up your own threads. (Threads are extremely expensive to create and have in memory, the thread pool shares threads between many requests so makes much more efficient use of a small number of them.)

    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Wednesday, April 2, 2008 3:23 PM
  • User242249085 posted

    How can i figure out if my web server IIS 6.0 supports http range header request?
     

    RTFM/Search/Forms/...

    Can i request for a specific range of a file size with BeginGetResponse() ? 

    Asynchronous has exactly the same approach for setting up the request; the differences is that it does not block the current thread. 

     

    EDIT: Hit post too soon. 

    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Thursday, April 3, 2008 2:58 AM

All replies

  • User242249085 posted

    My code below doesn't work.
     

    Could you describe in what way (symptoms etc.) it does not work.

    how can i request the thread to donwload
     

    You can perform a web request asynchronously (replace call to GetResponse with BeginGetResponse and provide a delegate to be called when response is available, you can also read the response stream asynchronously by using BeginRead/EndRead in place of Read). This all follows the established .NET asynchronous pattern. 

    All such asynchronous IO takes place via .NET's thread pool.

    Of you want to use a specific thread (one you have spun up yourself), simply perform all the processing (including initial request) on that thread. But generally the thread pool is a much easier solution. 

    Wednesday, April 2, 2008 3:42 AM
  • User-1702225963 posted

    Hi rjcox,

          The HttpWebRequest.AddRange(startbyte,endbyte) doesn't work.  The server returns the full range of stream. Please take a look at my code.

    private void btnDownload_Click(object sender, EventArgs e)

    {

    ////// number of downloading threads

    NoOfThread = 1;

    TotalSize = TotalFileSize();

    thrDownload =
    new Thread[NoOfThread];

    strRead = new Stream[NoOfThread];

    strWrite = new Stream[NoOfThread];if (NoOfThread == thrDownload.Length && NoOfThread == strRead.Length && NoOfThread == strWrite.Length)

    {

    Int64 EachFileSize = SplitFileSize(NoOfThread);

    //make request to the server to determine start point and end point of data streaming.

    //start the threading

    for (int i = 0; i < NoOfThread; i++)

    {

    Int64 StartPoint = (EachFileSize * (i));if (i == 0)

    {

    StartPoint = 0;

    strRead[i] = SendRequestToServer(StartPoint, EachFileSize - 1);

     

    }

    else if (i == 1)

    {

    StartPoint = EachFileSize;

    strRead[i] = SendRequestToServer(EachFileSize, ((EachFileSize * (i + 1)) - 1));

    }

    else if (i == NoOfThread - 1) //last request

    {

    strRead[i] = SendRequestToServer(StartPoint, ((EachFileSize * (i + 1))));

    }

    else

    {

    strRead[i] = SendRequestToServer(StartPoint, ((EachFileSize * (i + 1)) - 1));

    }

    //start multi-thread downloading process

    thrDownload[i] = new Thread(new ParameterizedThreadStart(Download));

    thrDownload[i].Start(i);

    }

    }

     

    }

     

    private Stream SendRequestToServer(object StartPoint,object EndPoint)

    {

     

    webRequest = (
    HttpWebRequest)WebRequest.Create(SOURCE_FILENAME);webRequest.Credentials = CredentialCache.DefaultCredentials;

    webRequest.ServicePoint.ConnectionLimit = 500;

    webRequest.AddRange("Ranges",Convert.ToInt32(StartPoint),Convert.ToInt32(EndPoint));

    webResponse = (HttpWebResponse)webRequest.BeginGetResponse();

    return webResponse.GetResponseStream();

    }

     

    private void Download(object OrderOfThread)

    {

    string Order = string.Empty;

    int i = (int)OrderOfThread;

    if (i < 9)

    {

    Order =
    "00" + (i + 1);

    }

    else

    {

    Order =
    "0" + (i + 1);

    }

    try

    {

    byte[] downBuffer = new byte[2048];

    int bytesSize = 0;

    strWrite[i] = new FileStream(@"C:\demo" + Order + ".exe", FileMode.Create, FileAccess.Write, FileShare.None);while ((bytesSize = strRead[i].Read(downBuffer, 0, downBuffer.Length)) > 0)

    {

    strWrite[i].Write(downBuffer, 0, bytesSize);

    if (i + 1 == NoOfThread)

    {

    this.Invoke(new UpdateProgessCallback(this.UpdateProgress), new object[] { strWrite[i].Length, TotalSize });

    }

     

    if (goPause == true)

    {

    break;

    }

    }

    }

    finally

    {

    // When the above code has ended, close the streams

    if (strRead[i] != null)

    {

    strRead[i].Close();

    strWrite[i].Close();

    }

    }

    }

    Wednesday, April 2, 2008 12:34 PM
  • User242249085 posted

    The HttpWebRequest.AddRange(startbyte,endbyte) doesn't work.  The server returns the full range of stream.
     

    Not all servers support range requests, you need to handle the case where the server does not.

    I would also strongly suggest using the IO thread pool  by using request.BeginGetResponse() rather than spinning up your own threads. (Threads are extremely expensive to create and have in memory, the thread pool shares threads between many requests so makes much more efficient use of a small number of them.)

    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Wednesday, April 2, 2008 3:23 PM
  • User-1702225963 posted

    Thanks rjcox,

         How can i figure out if my web server IIS 6.0 supports http range header request?  Can i request for a specific range of a file size with BeginGetResponse() ? 

    Wednesday, April 2, 2008 3:30 PM
  • User242249085 posted

    How can i figure out if my web server IIS 6.0 supports http range header request?
     

    RTFM/Search/Forms/...

    Can i request for a specific range of a file size with BeginGetResponse() ? 

    Asynchronous has exactly the same approach for setting up the request; the differences is that it does not block the current thread. 

     

    EDIT: Hit post too soon. 

    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Thursday, April 3, 2008 2:58 AM
  • User1269281531 posted

    I'm having the same problem -some servers supporting the range header request and some not. This example http://msdn.microsoft.com/en-us/library/7fy67z6d.aspx#Y1026 shows how to do it synchonously and it works for the www.contoso.com site that it uses.

    My site uses IIS7, and when running the sample code on that, it returns the full page. So looks like IIS7 doesn't support the range header. I'd guess therefore that IIS6 wouldn't either.

    Using the asynchonous method, can we just cancel the download when the desired number of bytes is reached? If so this would work for servers that don't support the range header. Of course, this would only work from the start of the file rather than part way through.

    Has anyone done this?

    Friday, May 27, 2011 9:26 AM
  • User-544877707 posted

    any thing replacement for

    webRequest.AddRange

    Tuesday, September 10, 2013 8:21 AM