locked
HttpWebRequest and CookieContainer - login to a website

    Question

  • Hi

    I am trying to log in to a website using CookieContainer and HttpWebRequest.

    I manage to log into the website, but then when I navigate to the next page I get thrown out. Using Fiddler, I saw that when I send the next request to the server, using the same CookieContainer, for some reason the cookie isn't sent. I have tried different approaches for this, and also some code samples I found over the web, but nothing works... what am I doing wrong?

    Here is my class code:

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using System.Net;
    using System.IO;
    
    namespace ShedateLogger
    {
      internal class tester
      {
        public void xxx()
        {
          CookieContainer cookieJar = new CookieContainer();
          CookieContainer cookieJar2 = new CookieContainer();
          
          HttpWebRequest req = (HttpWebRequest)HttpWebRequest.Create("http://www.shedate.co.il/login.asp");
          req.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705;)";
          req.Method = "POST";
          req.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
          req.Headers.Add("Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7");
          req.KeepAlive = true;
          req.Headers.Add("Keep-Alive: 300");
          req.AllowAutoRedirect = false;
     
          req.ContentType = "application/x-www-form-urlencoded";
    
          req.CookieContainer = cookieJar;
          
          String Username = "user";
          String PassWord = "pwd";
    
          StreamWriter sw = new StreamWriter(req.GetRequestStream());
          sw.Write("userEmail=" + Username + "&userPass=" + PassWord);
          sw.Close();
    
          HttpWebResponse response = (HttpWebResponse)req.GetResponse();
    
          StreamReader reader = new StreamReader(response.GetResponseStream(), Encoding.GetEncoding(1255));
          string tmp = reader.ReadToEnd();
    
          //Add cookies to CookieJar (Cookie Container)
          foreach (Cookie cookie in response.Cookies)
          {
            cookieJar.Add(new Cookie(cookie.Name.Trim(), cookie.Value.Trim(), cookie.Path, cookie.Domain));
            cookieJar2.Add(new Cookie(cookie.Name.Trim(), cookie.Value.Trim(), cookie.Path, cookie.Domain));
          }
          
          /// user page: the previous page redirects to this one...
          HttpWebRequest req2 = (HttpWebRequest)HttpWebRequest.Create("http://www.shedate.co.il/user.asp");
          req2.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705;)";
          req2.Method = "GET";
          req2.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
          req2.Headers.Add("Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7");
          req2.KeepAlive = true;
          req2.Headers.Add("Keep-Alive: 300");
          req2.CookieContainer = cookieJar;
    
          req2.ContentType = "text/html";
    
          HttpWebResponse response2 = (HttpWebResponse)req2.GetResponse();
    
          StreamReader reader2 = new StreamReader(response2.GetResponseStream(), Encoding.GetEncoding(1255));
          string tmp2 = reader2.ReadToEnd();
    
          response2.Close();
    
          /// profile page: this is the page where the server returns the wrong page since it doesnt get the cookies
          HttpWebRequest req3 = (HttpWebRequest)HttpWebRequest.Create("http://shedate.co.il/members/results.asp?isOnline=1&pageNum=5");//"http://shedate.co.il/users/profile.asp?userId=530537693");
          req3.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.0.3705;)";
          req3.Method = "GET";
          req3.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
          req3.Headers.Add("Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7");
          req3.KeepAlive = true;
          req3.Headers.Add("Keep-Alive: 300");
          req3.CookieContainer = cookieJar2;// new CookieContainer();
          //req3.CookieContainer = cookieJar;
          req3.AllowAutoRedirect = false;
    
          req3.ContentType = "text/html";
    
          response2 = (HttpWebResponse)req3.GetResponse();
    
          reader2 = new StreamReader(response2.GetResponseStream(), Encoding.GetEncoding(1255));
          tmp2 = reader2.ReadToEnd();
    
          response2.Close();
        }
      }
    }
    


    amazingwolf
    Tuesday, February 15, 2011 8:15 PM

Answers

  • I am struggling with this same problem, and from what I have discovered thus far, the cookie container seems to have been designed to store cookies from multiple servers at the same time, and be smart enough to only send the cookies for the current server (the one you are making the request to).

    The idea seems to be that your app would have one cookie container for all of its requests, and ... life is good.

    Except, they dont just use web server when determining whether or not to send the cookies.  It seems to only send cookies to the exact same URL that the original cookie came from.  So, in your example, if your second request was sent to "http://www.shedate.co.il/login.asp" then I think it would send the cookie.

    Which renders the entire process useless.  And, better than that, they actually wipe cookies that you add directly via headers because you are supposed to use the cookie container.

    I believe that there is a way to tell it that a cookie belongs to http://mysite/* but I havent found it yet.


    EDIT:  I think I have it working now. (at least the problem I was having is fixed.)

    after you get the response from your login, where you are for-each-ing your way through the cookies.  Dont do that.  do this instead...

    //Grab the cookie we just got back for this specifc page

    string cookie = cookieJar.GetCookieHeader(req.RequestUri);


    //put it back in the cookie container for the whole server

    cookieJar.SetCookies(new Uri("http://www.shedate.co.il/"), cookie);

    now, when you make requests, make sure and set the cookie container to your cookie jar (like you are doing)

    throw away the 2nd cookie jar.  you dont need it.

     

     

    • Proposed as answer by Brent Larsen Saturday, April 23, 2011 6:34 PM
    • Marked as answer by amazingwolf Thursday, September 15, 2011 9:33 PM
    Saturday, April 23, 2011 2:41 PM

All replies

  • Run a diff on your http traffic and the browser's traffic in fiddler.

    The following is signature, not part of post
    Please mark the post answered your question as the answer, and mark other helpful posts as helpful, so they will appear differently to other users who are visiting your thread for the same problem.
    Visual C++ MVP
    Tuesday, February 15, 2011 9:55 PM
  • I have done so. As I explained - for some reason the cookie is NOT sent on the last request from my code, even though I use the same cookie container.

    I have also tried a new cookie container and filled it with the cookies from the HttpWebResponse object - with no success.

    The browser IS sending the cookie, of course. For some reason my code fails to do the same.

     

    Any ideas?


    amazingwolf
    Wednesday, February 16, 2011 5:33 PM
  • Still cannot find a solution for this one... Can anyone help?

    amazingwolf
    Sunday, February 20, 2011 6:18 PM
  • I am struggling with this same problem, and from what I have discovered thus far, the cookie container seems to have been designed to store cookies from multiple servers at the same time, and be smart enough to only send the cookies for the current server (the one you are making the request to).

    The idea seems to be that your app would have one cookie container for all of its requests, and ... life is good.

    Except, they dont just use web server when determining whether or not to send the cookies.  It seems to only send cookies to the exact same URL that the original cookie came from.  So, in your example, if your second request was sent to "http://www.shedate.co.il/login.asp" then I think it would send the cookie.

    Which renders the entire process useless.  And, better than that, they actually wipe cookies that you add directly via headers because you are supposed to use the cookie container.

    I believe that there is a way to tell it that a cookie belongs to http://mysite/* but I havent found it yet.


    EDIT:  I think I have it working now. (at least the problem I was having is fixed.)

    after you get the response from your login, where you are for-each-ing your way through the cookies.  Dont do that.  do this instead...

    //Grab the cookie we just got back for this specifc page

    string cookie = cookieJar.GetCookieHeader(req.RequestUri);


    //put it back in the cookie container for the whole server

    cookieJar.SetCookies(new Uri("http://www.shedate.co.il/"), cookie);

    now, when you make requests, make sure and set the cookie container to your cookie jar (like you are doing)

    throw away the 2nd cookie jar.  you dont need it.

     

     

    • Proposed as answer by Brent Larsen Saturday, April 23, 2011 6:34 PM
    • Marked as answer by amazingwolf Thursday, September 15, 2011 9:33 PM
    Saturday, April 23, 2011 2:41 PM
  • Brent -

    Just stepped in here and found you answer. I also figured this out a few months back, after having a hard time with this issue - the web site I tried to access used both [www.domain.co.il] and [domain.co.il] for some reason, and I failed to see it.

    After figuring this out, the next step was to read the cookies and rewrite them under the current used domain in the cookie container.

     

    Thanks a lot for your answer! :-)


    amazingwolf
    Thursday, September 15, 2011 9:33 PM
  • I am doing same from 1st responce getting cookies and in next request for same user I want to post it back again to same server or different server, but problem is that  I am sending next request on next page, so how to get cookieContainer object declared at first page to next page with cookieJar...and what request need to send (like URL+session["prevoiuspagecookies"]) or what else: my current Url for next request from next page is :cookies stored in session from first page first responce and in next page set for second request
     cookieJar.SetCookies(new Uri("http://10.253.32.10:8080/52k-br/wps?"), Session["cookie"].ToString());
    cookies getting from first page from first responce:
     foreach (Cookie c in response.Cookies)
                      {
                          cookieJar.Add((new Cookie(c.Name.Trim(), c.Value.Trim(), c.Path, c.Domain)));
                         string  cookie = cookieJar.GetCookieHeader(request.RequestUri);
                          Session["cookie"] = cookie;
                      }



    --Himanshu Kulkarni Mark the thread as answered if the answer helps you. This helps others who have the same problem !



    --Himanshu Kulkarni, Mark the thread as answered if the answer helps you. This helps others who have the same problem !!!

    Tuesday, April 03, 2012 2:34 PM