Getting cookies using System.Net.Webclient class

Question

Subject almost says it all, im trying to get the appropriate cookies for the webpage im loading. The webpage source loads fine, but there doesn't seem to be a way to control/view/use the cookies i should be getting with the page. This makes it impossible for the program to visit pages where you need to be logged in to view. The login code is posted below, with different URL's that is:


System.Net.WebClient webcli = new System.Net.WebClient();
        System.Collections.Specialized.NameValueCollection namcol = new System.Collections.Specialized.NameValueCollection();
        System.Uri uri = new System.Uri("http://www.example.com/login2.php");
        namcol.Add("v","2");
        namcol.Add("login",loginBox.get_Text());
        namcol.Add("pass", passBox.get_Text());
        ubyte[] response = webcli.UploadValues(uri, "POST", namcol);

        listBox1.get_Items().Clear();
        listBox1.get_Items().Add(System.Text.ASCIIEncoding.get_ASCII().GetString(response));
        Showsource(response);

Showsource does just that, show the source. This all works and i can verify that it DOES login as long as the values in namcol are correct. The only problem is that its impossible to get any further without the cookies. How do i get them? And how do i use them to verify that im logged in?

If anyone knows or has a hint/tip/lead, don't hesitate to post.

Answer 1

Okay, I fixed it myself after some puzzling.

I don't think you can do it with the WebClient class, so I used the Webrequest class instead. The Visual J# index has enough information on how to use it, so that wasn't a problem. For people who want to know how to use it the code is posted below. This example let's you login to a site that uses cookies to identify you. I've seen multiple questions about this kind of thing, well here it is:

Code Snippet

listBox1.get_Items().Clear();

        System.Net.HttpWebRequest webreq = ((System.Net.HttpWebRequest)(System.Net.WebRequest.Create("http://uni11.ogame.org/game/reg/login2.php")));
        System.Net.CookieContainer cookies = new System.Net.CookieContainer();
        ubyte[] postdata = System.Text.Encoding.get_Default().GetBytes("v=2&universe=uni11.ogame.org&login=xxxx&pass=xxxx&button.x=16&button.y=4");

        webreq.set_CookieContainer(cookies);
        webreq.set_UserAgent("Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)");
        webreq.set_Method("POST");
        webreq.set_Credentials(System.Net.CredentialCache.get_DefaultCredentials());
        webreq.set_ContentLength(postdata.length);
        webreq.set_ContentType("application/x-www-form-urlencoded");

        System.IO.Stream webstream = webreq.GetRequestStream();
        webstream.Write(postdata, 0, postdata.length);
        webstream.Close();

        System.Net.WebResponse response = webreq.GetResponse();
        listBox1.get_Items().Add(((System.Net.HttpWebResponse)response).get_StatusDescription());
        webstream = response.GetResponseStream();
        System.IO.StreamReader reader = new System.IO.StreamReader(webstream);
        String responseFromServer = reader.ReadToEnd();
        listBox1.get_Items().Add(responseFromServer);
        reader.Close();
        webstream.Close();
        response.Close();

        listBox1.get_Items().Add("Cookies #:" + String.valueOf(cookies.get_Count()));
        if( cookies.GetCookies(new System.Uri("http://uni11.ogame.org/game/reg/login2.php")).get_Item(0) != null)
            listBox1.get_Items().Add("Value of cookie 0:" + cookies.GetCookies(new System.Uri("http://uni11.ogame.org/game/reg/login2.php")).get_Item(0).get_Value());
        if (cookies.GetCookies(new System.Uri("http://uni11.ogame.org/game/reg/login2.php")).get_Item(1) != null)
            listBox1.get_Items().Add("Value of cookie 1:" + cookies.GetCookies(new System.Uri("http://uni11.ogame.org/game/reg/login2.php")).get_Item(1).get_Value());
        if (cookies.GetCookies(new System.Uri("http://uni11.ogame.org/game/reg/login2.php")).get_Item(2) != null)
            listBox1.get_Items().Add("Value of cookie 2:" + cookies.GetCookies(new System.Uri("http://uni11.ogame.org/game/reg/login2.php")).get_Item(2).get_Value());

        String sessionid = listBox1.get_Items().get_Item(1).toString();
        sessionid = sessionid.substring(sessionid.indexOf("session=")+"sessionid".length() - 1,sessionid.lastIndexOf("&"));
        listBox1.get_Items().Add(sessionid);

        webreq = ((System.Net.HttpWebRequest)(System.Net.WebRequest.Create("http://uni11.ogame.org/game/index.php?page=overview&session=" + sessionid.toString() + "&lgn=1")));
        webreq.set_CookieContainer(cookies);
        webreq.set_UserAgent("Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)");
        webreq.set_Credentials(System.Net.CredentialCache.get_DefaultCredentials());
        response = webreq.GetResponse();
        listBox1.get_Items().Add(((System.Net.HttpWebResponse)response).get_StatusDescription());
        webstream = response.GetResponseStream();
        reader = new System.IO.StreamReader(webstream);
        responseFromServer = reader.ReadToEnd();
        Showsource(responseFromServer);
        reader.Close();
        webstream.Close();
        response.Close();

 

Code Snippet

   private void Showsource(String response)
    {
        char[] chalis = new char[1];
        chalis[0] = '\n';
        String[] source = response.Split(chalis);

        for (int X = 0; X < source.length; X++)
        {
            source[X].TrimEnd(chalis);
            listBox1.get_Items().Add(source[X]);
        }
    }

You can see that it gets the cookies first using the POST method, then it saves them and even shows them in a listbox. After which it connects to the site BEHIND the login form and gets that page, which it will also show in the textbox. Don't expect a nice webpage though, it only downloads the source code of the page.

To get the right parameters and address to POST stuff to, I used the Tamper Data add-on for FireFox. It shows exactly what data goes in and out of your browser.

Also, if you compile this example, it will work on the given site aslong as you replace the x's with your own username and password ( not encrypted in any form ).

Answer 2

Thanks for your post!

You can still use WebClient and retrieve session cookie from the response Headers and then reuse it on sub-sequent requests with a new WebClient instance. WebClient doesn’t provide a collection for cookies so you have to generate a proper string value for the Cookie header.

Here is the code (VB.NET)

After your first WebClient request came back the cookie list can be retrieved from .ResponseHeaders collection:

Dim cookies as String  =  webClientIntsance1.ResponseHeaders(“Set-Cookie”)

The cookies string may look something like this for the ASP.NET session (IIS web server) cookie for example (in this case no other cookies have been sent to the browser):

"ASP.NET_SessionId=tdqlpzvdwlcc2cjq4rft4bz1; path=/; HttpOnly"

       Different servers generate different session cookie name and format. Weblogic server cookie may look something like this:

       JSESSIONID=KSLTP23Ywlb1S2yvvCbQnhRND2CLRlRNykTXJDwlwTBGLSY4dq7B!-583862679; path=/

Then with your next WebClient request you have to add this session cookie to the .Headers collection:

Dim webClientIntsance2 as New WebClient

webClientIntsance2.Headers.Add(“Cookie”, "ASP.NET_SessionId=tdqlpzvdwlcc2cjq4rft4bz1; path=/; HttpOnly")

Note the difference in the name of the cookie header for request and response. The response header is called Set-Cookie, while request header is Cookie.

Hope this helps!

Svetlana Shelkova

Answer 3

You can still use CookieContainer and CookieCollection with System.Net.WebClient if you have your own webclient class and override GetWebRequest/GetWebResponse methods:

Public Class MyWebClient
    Inherits System.Net.WebClient

    Private _CookiesRequest As System.Net.CookieContainer
    Private _CookiesResponse As System.Net.CookieCollection

    Protected Overrides Function GetWebRequest(ByVal address As System.Uri) As System.Net.WebRequest
        Dim request As System.Net.HttpWebRequest = DirectCast(MyBase.GetWebRequest(address), System.Net.HttpWebRequest)
        request.CookieContainer = _CookiesRequest
        Return request
    End Function

    Protected Overrides Function GetWebResponse(ByVal request As System.Net.WebRequest, ByVal result As System.IAsyncResult) As System.Net.WebResponse
        Dim response As System.Net.HttpWebResponse = DirectCast(MyBase.GetWebResponse(request, result), System.Net.HttpWebResponse)
        _CookiesResponse = response.Cookies
        Return response
    End Function
End Class

Svetlana Shelkova

 

Answer 4

Hey nice posts thank u

but my question is a little different 

what if we use the page say xyz.com for one thing and then after a submit button we redirect to the same page and add some more cookies say httponly then the code does not work. it only bring those cookies set on the page load