none
Httpwebrequest not sending cookies on Post RRS feed

  • Question

  • Hi, I’m writing a http module to work as a reverse proxy, i.e. receives a request from a browser, sends it on to the target site, receives a response and sends that back to the browser.

    Its all working fine, except for a problem with forwarding cookies from the browser request to the target site on a Post. All headers and form data are correct on the outgoing request, but no cookies are included.

    I’ve run fiddler on both the request from the browser to IIS and the outgoing httpwebrequest and proven this to be the case. Running the module in debug shows that the cookies are found in the request from the browser and successfully placed in the cookiecontainer of the httpwebrequest, but they just don’t appear in the actual request sent out.

    If I hack (in debug) the outgoing request method to a Get, then they go, but they don’t go for a Post.

    I’ve also tracked the request/response from a browser direct to the target site using Fiddler, and the request seems identical in all three cases (browser to target, browser to my IIS module, IIS module to target), except that the IIS module to target omits the cookies.

    Here’s the code (VB.Net, and tried in 2.0 and 4.5):

    ' set up the request to the target Dim reqTarget As System.Net.HttpWebRequest reqTarget = CType(System.Net.HttpWebRequest.Create(strTargetURL & strTargetPath & qstring), System.Net.HttpWebRequest) ' copy relevant info, cookies etc from the application request to the target request CopyAppRequest(application.Context.Request, reqTarget) ' send the request and get the response Dim rspTarget As System.Net.HttpWebResponse = CType(reqTarget.GetResponse(), System.Net.HttpWebResponse) Private Sub CopyAppRequest(ByRef reqApp As System.Web.HttpRequest, ByRef reqTarget As System.Net.HttpWebRequest) ' copy over the headers For Each key As String In reqApp.Headers.AllKeys Select Case key Case "Host", "Accept-Encoding", "Expect", "Authorization", "If-Modified-Since" ' not sure if we need to process these Case "Connection" reqTarget.Connection = reqApp.Headers(key) Case "Content-Type" reqTarget.ContentType = reqApp.Headers(key) Case "Accept" reqTarget.Accept = reqApp.Headers(key) Case "Referer" reqTarget.Referer = reqApp.Headers(key) Case "User-Agent" reqTarget.UserAgent = reqApp.Headers(key) Case "Cookie" ' do nothing, cookies are handled below.. Case Else reqTarget.Headers.Add(key, reqApp.Headers(key) End Select Next reqTarget.Method = reqApp.HttpMethod reqTarget.AllowAutoRedirect = False If reqTarget.Method = "POST" Then reqTarget.ContentLength = reqApp.ContentLength Dim datastream() As Byte = System.Text.Encoding.UTF8.GetBytes(reqApp.Form.ToString) reqTarget.ContentLength = datastream.Length Dim requestwriter As System.IO.Stream = reqTarget.GetRequestStream requestwriter.Write(datastream, 0, datastream.Length) requestwriter.Close() requestwriter.Dispose() End If Dim CookieJar As New System.Net.CookieContainer reqTarget.CookieContainer = CookieJar For Each key As String In reqApp.Cookies.AllKeys Dim tgtCookie As New System.Net.Cookie With tgtCookie .Name = reqApp.Cookies.Item(key).Name .Value = reqApp.Cookies.Item(key).Value .Domain = ".domain.com" .Path = "/" .Expires = DateAdd(DateInterval.Month, 1, System.DateTime.Now) .HttpOnly = True End With CookieJar.Add(tgtCookie) Next End Sub

    Note: the domain I’m trying to reach is in the form abc.domain.com (i.e. it’s a subdomain, and no www), the reason I’ve tried the .domain.com form is that is the form used in the cookies that are received in the response. I’ve also tried other combinations such as abc.domain.com, .abc.domain.com, etc.

    Also I’ve tried creating a Uri object and using that method to add the cookie into the cookiecontainer as well as creating a cookiecollection.

    I’ve tried this as well to check that the cookies are set up correctly:

     

    Dim strCookie As String = reqTarget.CookieContainer.GetCookieHeader(reqTarget.RequestUri)
    
    ' send the request and get the response
    rspTarget = CType(reqTarget.GetResponse(), System.Net.HttpWebResponse)

    Strcookie has the cookies I want in it, but they’re not showing in the request in Fiddler.

    I’ve tried everything I can think of and can find on forums…. Anyone got any suggestions? I suspect I’ve missed something obvious!

    Of course, any other comments on how the code above can be improved will be appreciated.

    Thanks.


    • Edited by RSdraig Friday, May 23, 2014 8:36 AM better formatting (indent) of code
    Thursday, May 22, 2014 12:43 PM

Answers

  • Ok, I found the issue...

    I was using Fiddler to see what was going on with http, and Fiddler was correct. However, when I used Wireshark, I found that the http request was being sent earlier than I thought, and only on the Post.

    It turned out that the requestwriter.write line caused the http request to be sent, not the GetResponse (as is the case with Get). So, anything I changed in the httpwebrequest after the requestwriter.write didn't get sent.

    The fix - I just moved all the header and cookie set up above the requestwriter.write and it all worked.

    How frustrating, but at least its fixed now :)

    If anyone has any feedback on whether I've got something wrong that's causing this to happen, please let me know.

    • Marked as answer by RSdraig Sunday, May 25, 2014 12:11 PM
    Sunday, May 25, 2014 12:11 PM

All replies

  • Hello,THe problem's that you can't just set the cookiecontainer to have the cookies from the response, since it's a new request we need to set the domain that the cookies belong to as well (.net is not assuming that the domain is the one from the URI in the Request object)

    so, we need to do something like this when setting the cookies :

    request.CookieContainer = new CookieContainer();
    request.CookieContainer.Add(new Uri("http://localhost"), originalResponse.Cookies);

    Just as another note, i was having a Path problem when setting the cookies.. getting an error like "The 'Path'=/MyApp part of the cookie is invalid". I solved this by setting the cookies' path to nothing before adding them (and making them valid in the whole domain)

       for (int i = 0; i < originalResponse.Cookies.Count; i++)
       {
            originalResponse.Cookies[i].Path = String.Empty;
       }
    Friday, May 23, 2014 6:18 AM
  • Thanks for this.

    I think that this part of the code is equivalent to what you're saying:

    For Each key As String In reqApp.Cookies.AllKeys
    
    	Dim tgtCookie As New System.Net.Cookie
     	With tgtCookie 
    		.Name = reqApp.Cookies.Item(key).Name
    		.Value = reqApp.Cookies.Item(key).Value
    		.Domain = ".domain.com"
    		.Path = "/"
     		.Expires = DateAdd(DateInterval.Month, 1, System.DateTime.Now)
    		.HttpOnly = True
     	End With
    	CookieJar.Add(tgtCookie)
    Next
    

    I tried your suggestion, only I can't allocate the cookies from the original request as its a system.web.httpcookiecollection whereas the outgoing request is a system.net.cookiecollection. Note that I'm transferring cookies from the incoming request object (from the browser) to the outgoing request object (to the target site).

    However, I changed Cookie jar to a system.net.cookiecollection:

    Dim CookieJar As New System.Net.CookieCollection

    and after moving the cookies over (as above), did what you suggest and made doubly sure that the path settings were blank:

            For i As Integer = 0 To CookieJar.Count - 1
                CookieJar.Item(i).Path = ""
            Next

    And then, instead of just assigning the cookiejar to the outgoing request (reqTarget.CookieContainer = CookieJar) I used the Add method:

    reqTarget.CookieContainer.Add(New Uri("http://abc.domain.com"), CookieJar)

    Same outcome. When I look in Fiddler, the outgoing request doesn't have the cookies in it, yet when I look at strCookie from:

    Dim strCookie As String = reqTarget.CookieContainer.GetCookieHeader(reqTarget.RequestUri)
    

    It has the cookies in it, which seems to confirm that they're correctly set up in the request object (well, obviously not since they're not being sent!).

    Again, the cookies are sent if the method is a GET, but not a POST!

    I'm really stuck on this one, so any suggestions from anyone will be greatly appreciated.

    Thanks

    Friday, May 23, 2014 8:46 AM
  • Ok, I found the issue...

    I was using Fiddler to see what was going on with http, and Fiddler was correct. However, when I used Wireshark, I found that the http request was being sent earlier than I thought, and only on the Post.

    It turned out that the requestwriter.write line caused the http request to be sent, not the GetResponse (as is the case with Get). So, anything I changed in the httpwebrequest after the requestwriter.write didn't get sent.

    The fix - I just moved all the header and cookie set up above the requestwriter.write and it all worked.

    How frustrating, but at least its fixed now :)

    If anyone has any feedback on whether I've got something wrong that's causing this to happen, please let me know.

    • Marked as answer by RSdraig Sunday, May 25, 2014 12:11 PM
    Sunday, May 25, 2014 12:11 PM
  • awesome... this just saved me a day of googling and finding out why cookies were not supported with the request.
    Tuesday, May 8, 2018 11:52 PM