locked
HttpWebRequest and HttpWebResponse question RRS feed

  • Question

  • User-507568172 posted

    Hi all,

    I'm trying to create a window application that does the following programically:

    1. Login to a web page (example: http://www.abc.com/login.aspx)
    2. When login successful, i then want to go to the web page that i desire. (Example: http://www.abc.com/customers.aspx)
      And grab all the data in the aspx page and store it in my local db.

    I've google around and many of the examples are using HttpWebRequest and HttpWebResponse classes,
    i've tried some of the samples and i manage to make the samples run, but i have no idea whether have i login successfully or not.
    I keep getting the login page's html tags after the whole method is completed at line 25 string htmlText = reader.ReadToEnd();

    Thanks in advance :)

    Below are my codes:
    Window application Code behine 

    1            public void GetTest()
    2            {
    3                try
    4                {
    5                    string strPost = "Username=1&Password=1";
    6                    string url = "http://localhost:50555/WebSite1/DummyLogin.aspx";
    7                    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
    8                    request.ContentType = "text/html";
    9                    request.Method = "POST";
    10                   request.AllowAutoRedirect = false;
    11                   request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 (.NET CLR 3.5.30729)";
    12                   request.ProtocolVersion = new Version(1, 0);
    13                   request.Accept = "*.*";
    14                   request.CookieContainer = new CookieContainer();
    15                   //request.Headers.Add("Username", "1");
    16                   //request.Headers.Add("Password", "1");
    17                   Stream requestStream = request.GetRequestStream();
    18                   byte[] postBytes = Encoding.ASCII.GetBytes(strPost);
    19                   requestStream.Write(postBytes, 0, postBytes.Length);
    20                   requestStream.Close();
    21   
    22                   HttpWebResponse response = (HttpWebResponse)request.GetResponse();
    23                   StreamReader reader = new StreamReader(response.GetResponseStream());
    24                   rtxtResult.AppendText(response.StatusCode.ToString());
    25                   string htmlText = reader.ReadToEnd();
    26                   rtxtResult.AppendText(htmlText); 
    27               }
    28               catch (Exception ex)
    29               {
    30                   rtxtResult.AppendText(ex.Message);
    31               }
    32           }
    

     

    Dummy Login aspx page

     

    <%@ Page Language="C#" AutoEventWireup="true" CodeFile="DummyLogin.aspx.cs" Inherits="DummyLogin" %>
    
    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
    
    <html xmlns="http://www.w3.org/1999/xhtml">
    <head runat="server">
        <title>Untitled Page</title>
    </head>
    <body>
        <form id="form1" runat="server">
        <div>
        
            <input type="text" name="Username" id="txtUsername" runat="server" />
            <br />
            <input type="text" name="Password" id="txtPassword" runat="server" />
            <br />
            <asp:Button ID="btnLogin" runat="server" Text="Login" 
                onclick="btnLogin_Click" />
        </div>
        </form>
    </body>
    </html>
     

     Dummy login page code behind
     

        protected void btnLogin_Click(object sender, EventArgs e)
        {
            if (txtUsername.Value == "1" && txtPassword.Value == "1")
                Response.Redirect("testDropdown.aspx");        
                
        }
     

     

    Tuesday, December 23, 2008 11:16 PM

Answers

  • User-507568172 posted

    Hi SyntaxC4,

    after reading your suggestion regarding the hidden field, i carefully look into my codes and running the website with firebug, I've found the problem with my codes. Just sharing what i found in case some newbie like me got stuck with this.

    1. __VIEWSTATE={0}&txtUsername={1}&txtPassword={2}&LoginButton=Login (The 2 text with bold should be Username and Password) [:$]

    Thank you SyntaxC4 for your time and sharing. :)

    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Friday, December 26, 2008 7:17 PM

All replies

  • User783383449 posted

    Hi Pandama,

    Maybe this forum can be of use: http://social.msdn.microsoft.com/forums/en-US/Offtopic/thread/cd935f8e-f735-4650-8eb6-90c19fe830b3

    Hope this helps.

    Wednesday, December 24, 2008 2:30 AM
  • User-507568172 posted

    hi SyntaxC4,

    thanks for replying, i've been trying out the same method provided in the link you share.

    The content (string content = reader.ReadToEnd(); ) that I keep getting the is the html tags from the login page.

    I just couldn't get the content of the main page which is shown after login.

     

    thanks again in advance :)

     

    Wednesday, December 24, 2008 5:12 AM
  • User783383449 posted

    Hi pandama,

    The url you are trying to send to has been moved.  https://r.abc.go.com/abc/register/login  is the new location of the login page.  if you view the source on this page you will notice a bunch of hidden variables that you might have to pass along with your login.

    Hope this helps.

    Wednesday, December 24, 2008 11:24 AM
  • User-507568172 posted

    hi SyntaxC4,

    thank you for replying my post, please accept my appologies for my previous post which cause so many misleading info regarding my question.
    the url with www.abc.com and others are just an example i gave to explain my senario.
    It is hard for me to provide you the actual link that i'm working on because the data there is my company's data.

    Alright, i'll try my best to explain my situation.

    What i want to create is basically a C# win form web crawler that login to a web site and go to a sub page to crawl all the data from the gridview.

    i've tried the same in the link that you provided http://social.msdn.microsoft.com/forums/en-US/Offtopic/thread/cd935f8e-f735-4650-8eb6-90c19fe830b3,
    after running the code, the content i got at the last line which is ( string content = reader.ReadToEnd();) is still the login page's content.

    How do i get the content of the sub page after i login?
    (Example: after i successfully login, i want to get the content in sub page A, and then later go to sub page B and get the it's content)

    Below are the codes that i create a dummy web project to test out,

    1                string url = http://localhost:50555/WebSite1/DummyLogin.aspx;
    2 string url2 = http://localhost:50555/WebSite1/MemberList.aspx; 3 HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url); 4 req.AllowAutoRedirect = false; 5 req.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 (.NET CLR 3.5.30729)"; 6 req.ProtocolVersion = new Version(1, 0); 7 req.Accept = "*.*"; 8 req.CookieContainer = new CookieContainer(); 9 StreamReader rdr = new StreamReader(req.GetResponse().GetResponseStream()); 10 string responseData = rdr.ReadToEnd(); 11 rdr.Close(); 12 13 string viewState = ExtractViewState(responseData); 14 string postData = string.Format("__VIEWSTATE={0}&txtUsername={1}&txtPassword={2}&LoginButton=Login", 15 viewState, "1", "1"); 16 17 HttpWebRequest req2 = (HttpWebRequest)WebRequest.Create(url2); 18 req2.Method = "POST"; 19 req2.ContentType = "application/x-www-form-urlencoded"; 20 req2.AllowAutoRedirect = true; 21 req2.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.5) Gecko/2008120122 Firefox/3.0.5 (.NET CLR 3.5.30729)"; 22 req2.ProtocolVersion = new Version(1, 0); 23 req2.Accept = "*.*"; 24 req2.ContentLength = postData.Length; 25 req2.CookieContainer = new CookieContainer(); 26 27 StreamWriter writer = new StreamWriter(req2.GetRequestStream()); 28 writer.Write(postData); 29 writer.Close(); 30 StreamReader rdr2 = new StreamReader(req2.GetResponse().GetResponseStream()); 31 rtxtResult.AppendText(rdr2.ReadToEnd()); 32 rdr2.Close(); 33 req2.GetResponse().Close();

     

    Please let me know if you still need more information.

    Thanks again in advance :)

    Thursday, December 25, 2008 9:25 PM
  • User-507568172 posted

    Hi SyntaxC4,

    after reading your suggestion regarding the hidden field, i carefully look into my codes and running the website with firebug, I've found the problem with my codes. Just sharing what i found in case some newbie like me got stuck with this.

    1. __VIEWSTATE={0}&txtUsername={1}&txtPassword={2}&LoginButton=Login (The 2 text with bold should be Username and Password) [:$]

    Thank you SyntaxC4 for your time and sharing. :)

    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Friday, December 26, 2008 7:17 PM
  • User1525643254 posted

    In window application it is very easy 

    try to use web browser control  it also have various event listeners

    See the below code

    private void Login()
    {
    WebBrowser webBrowser1 = new WebBrowser();
    webBrowser1.DocumentCompleted += new WebBrowserDocumentCompletedEventHandler(b_DocumentCompleted);
    webBrowser1.Navigate(url);
    }

    private void b_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
    {
    WebBrowser b = sender as WebBrowser;
    string response = b.DocumentText;

    int index = response.IndexOf("form");

    if (index != -1)
    {
    HtmlElement userName = b.Document.GetElementById("L1");
    userName.SetAttribute("value", username);

    HtmlElement password1 = b.Document.GetElementById("L2");
    password1.SetAttribute("value", password);

    HtmlElementCollection logonForm = b.Document.GetElementsByTagName("button");
    logonForm[0].InvokeMember("click");

    // unregisters the first event handler
    // adds a second event handler
    b.DocumentCompleted -= new WebBrowserDocumentCompletedEventHandler(b_DocumentCompleted);
    b.DocumentCompleted += new WebBrowserDocumentCompletedEventHandler(b_DocumentCompleted2);
    }
    }

    private void b_DocumentCompleted2(object sender, WebBrowserDocumentCompletedEventArgs e)
    {
    WebBrowser b = sender as WebBrowser;
    string response = b.Document.Body.InnerHtml;

    }

    Thursday, August 22, 2013 6:31 AM