locked
POST WebRequest

    Question

  • Hello,

    I'm trying to log in to a website programmatically. My first thought it was to construct a string with the requested values . Example for the following login form :



      <!-- login form -->
      <form action="/login.php" method="post">
    <input type="hidden" name="action" value="login">
    <input type="hidden" name="url" value="/forum">
      <script type="text/javascript" src="clientscript/vbulletin_md5.js"></script>
      <table cellpadding="0" cellspacing="3" border="0">
      <tr>
       <td class="smallfont">User Name</td>
       <td><input type="text" class="button" name="username" id="navbar_username" size="10" accesskey="u" tabindex="1" value="User Name" onfocus="if (this.value == 'User Name') this.value = '';" /></td>
       <td class="smallfont" colspan="2" nowrap="nowrap"><label for="cb_cookieuser_navbar"><input type="checkbox" name="perm" value="1" tabindex="3" id="cb_cookieuser_navbar" accesskey="c" checked="checked" />Remember Me?</label></td>
      </tr>
      <tr>
       <td class="smallfont">Password</td>
       <td><input type="password" class="button" name="password" size="10" accesskey="p" tabindex="2" /></td>
       <td><input type="submit" class="button" value="Log in" tabindex="4" title="Enter your username and password in the boxes provided to login, or click the 'register' button to create a profile for yourself." accesskey="s" /></td>
      </tr>
      </table>
      <input type="hidden" name="s" value="afaae00e27563b2088e35d6efa94597d" />
      <input type="hidden" name="do" value="login" />
      <input type="hidden" name="forceredirect" value="1" />  
      <input type="hidden" name="vb_login_md5password" />
      </form>
      <!-- / login form -->

    I think you can do something like this :
    www.site.com/login.php?username=myUser&password=myPass&do=login

    The problem is this is not working ( a screen appears with wrong password). There are other issues here like the encoding with clientscript/vbulletin_md5.js . I suppose a hashcode is generated with every page accessing and is stored in a cookie.

    I also thought to construct a POST message that I could send to www.site.com/login.php from my C# application , but I don't know how to do it . I used HttpWebRequest (instead of WebRequest ) because has support for cookies.

     

     

    So , it should be something like this (see the C# example) :

    http://msdn2.microsoft.com/en-us/library/debx8sh9.aspx

    where I replace the byteArray  from here :   
                dataStream.Write (byteArray, 0, byteArray.Length);

    with one that has the POST message . The problem is how the POST request should look like , given the <!-- login form --> above.

    Thank you


     

     

    Monday, September 11, 2006 1:58 PM

All replies

  • There are tons of examples on the net
    I searched for a few seconds and I got
    http://www.code101.com/Code101/DisplayArticle.aspx?cid=82

     

     

    Monday, September 11, 2006 2:26 PM
    Moderator
  • There may be tons of them , but I can not find a good one.

    This is my code for HttpWebRequest:


    CookieContainer cookies = new CookieContainer();

    mystr = BuildString();

    ASCIIEncoding encoding = new ASCIIEncoding();

    data = encoding.GetBytes(mystr);

    HttpWebRequest wr = (HttpWebRequest)WebRequest.Create("http://www.site.com/forum/");

    wr.CookieContainer = cookies;

    wr.KeepAlive = true;

    wr.ContentLength = data.Length;

    wr.Credentials = CredentialCache.DefaultCredentials;

    wr.ContentType = "application/x-www-form-urlencoded";

    wr.Method = "POST";

    Stream newStream;

    newStream = wr.GetRequestStream();

    newStream.Write(data, 0, data.Length);

    newStream.Close();

    HttpWebResponse wresponse =(HttpWebResponse) wr.GetResponse();

    StreamReader reader = new StreamReader(wresponse.GetResponseStream());

     

    //And BuildString(); method that builds my POST form , built from <log-in form>

    private string BuildString()

    {

    StringBuilder strb = new StringBuilder();

    strb.Append("<form action=\"/login.php\" method=\"post\">");

    strb.Append("<input type=\"hidden\" name=\"action\" value=\"login\">");

    strb.Append("<input type=\"hidden\" name=\"url\" value=\"/forum\">");

    strb.Append("<input type=\"hidden\" name=\"s\" value=\"\" />");

    strb.Append("<input type=\"hidden\" name=\"do\" value=\"login\" />");

    strb.Append("<input type=\"hidden\" name=\"forceredirect\" value=\"1\" /> ");

    strb.Append("</form>");

    return strb.ToString();

    }


    Obviously this doesn't work . I don't really get what is the problem .

    Thank you

    Monday, September 11, 2006 6:15 PM
  • Are you serious? What you are sending is the HTML not the form POST.
    The HTML is for the browsers. Browsers don't send HTML back to the server for POST
    They just send Name=Value pairs and they have to be URL encoded.

    You need to get some know-how on how the browsers send the form back to the server
    I suggest get Ethereal and look at the packets and you have a good idea of what you need to send
    blindly sending html back to the sever is obvioulsy a bad idea

    Monday, September 11, 2006 6:40 PM
    Moderator
  • Don't blame me . This is not my known-best part of programming , that is why I'm asking you . I just saw something related ( http://www.netomatix.com/development/postrequestform.aspx ) and had an idea.

    I even tried this way : http://www.code101.com/Code101/DisplayArticle.aspx?cid=82

     

    Monday, September 11, 2006 6:49 PM
  • OK so get the name and value pairs
    Like
    UserName=Blah
    Password=Blah

    Then URLEncode each of those
    So it looks like
    foreach name value pair.
      string = urlencode(name) + "=" + urlencode(value)

    Then get the bytes
    bytes = utf8encode(string)

    then set the content type to "application....;charset=utf8

    then post it

    Monday, September 11, 2006 7:24 PM
    Moderator
  • Ok . That sofware is really interesting . I found that he sends :

    action=login&url=2Fforum&username=myuser&perm=1&password=mypassword&s=&do=login&forceredirect=1&vb_login_md5passwor

    d=

    I tried to send this , but the same problem . I see that when I first accessed the site (from IE) he makes a sessionHash and stores it in a cookie . Then when I submit my user and password , the browser POSTs that datas like above and he uses a cookie with bbsessionhash. That bbsessionhas is generated on the first call of the site , but not on the first GET method . So , when I submit my user and pass , he uses POST and the cookie with bbsessionhash . I think I first need to request the page , store that cookie , then POST my datas using the same CookieContainer . Any idea ?

    Thanks

    Monday, September 11, 2006 8:01 PM
  • If I understand you correctly I had to do something very similar.  As I understand it you are attempting to log in to a webpage using your application. Unfortunatel.y rather than using basic authentication or something like that it uses a form based login on the web page.  What I had to do was login to this web page and retrieve two cookies out of the set cookies header that followed some code is below

     

    Dim strPortal As String = AppSettings.PortalURL

    Dim myHttpWebRequest As HttpWebRequest = CType(WebRequest.Create(strPortal), _

    HttpWebRequest) 'Request

    Dim encodedData As New System.Text.UTF8Encoding

    'data to be entered into fields

    Dim postData As String = String.Format("username={0}&password={1}&submit.x=38&submit.y=5" _

    , userId, password)

    'array of bytes to transmit field data

    Dim byteArray As Byte() = encodedData.GetBytes(postData)

    Dim cookies As New CookieContainer

    Dim newStream As Stream

    'setup request object

    myHttpWebRequest.Method = "POST"

    myHttpWebRequest.ContentType = "application/x-www-form-urlencoded|application/xml"

    myHttpWebRequest.AllowWriteStreamBuffering = True

    myHttpWebRequest.Timeout = CType(New TimeSpan(0, 0, 60).TotalMilliseconds, Integer)

    myHttpWebRequest.ProtocolVersion = HttpVersion.Version10

    'write field data to the request objects stream (gives the web page the data)

    newStream = myHttpWebRequest.GetRequestStream()

    Try

    newStream.Write(byteArray, 0, byteArray.Length)

    Catch ex As Exception

    Throw

    Finally

    newStream.Close()

    End Try

    'get the response

    Dim myHttpWebResponse As HttpWebResponse = CType(myHttpWebRequest.GetResponse(), HttpWebResponse)

    'create a cookie collection to house the cookies

    'and put the response cookies in it HERE IS WHERE I START THE COOKIE STUFF ANYTHING ABOVE IS LOGGING INTO THE WEB SITE!

     

    Dim header As String = myHttpWebResponse.Headers.Item("Set-Cookie")

    'Get Cookie 1

    Dim intCookieNameEnd, intCookieValueEnd As Integer

    intCookieNameEnd = header.IndexOf("=")

    intCookieValueEnd = header.IndexOf(";")

    Dim strName, strValue As String

    strName = header.Substring(0, intCookieNameEnd)

    strValue = header.Substring(intCookieNameEnd + 1, intCookieValueEnd - 1 - intCookieNameEnd)

    Dim cookie1 As New Cookie(strName, strValue)

    cookie1.Domain = myHttpWebRequest.RequestUri.Host

    header = header.Remove(0, intCookieValueEnd + 1)

    'Getting path of cookie 1

    intCookieNameEnd = header.IndexOf("=")

    intCookieValueEnd = header.IndexOf(";")

    cookie1.Path = header.Substring(intCookieNameEnd + 1, intCookieValueEnd - 1 - intCookieNameEnd)

    header = header.Remove(0, intCookieValueEnd + 1)

    Dim intSecure As Integer = header.IndexOf(",")

    'Getting whether cookie is secure

    If header.Substring(0, intSecure).Trim.Equals("Secure") Then

    cookie1.Secure = True

    End If

    header = header.Remove(0, intSecure + 1)

    'Get cookie2

    intCookieNameEnd = header.IndexOf("=")

    intCookieValueEnd = header.IndexOf(";")

    strName = header.Substring(0, intCookieNameEnd)

    strValue = header.Substring(intCookieNameEnd + 1, intCookieValueEnd - 1 - intCookieNameEnd)

    Dim cookie2 As New Cookie(strName, strValue)

    cookie2.Domain = myHttpWebRequest.RequestUri.Host

    header = header.Remove(0, intCookieValueEnd + 1)

    'Getting path of cookie 2

    intCookieNameEnd = header.IndexOf("=")

    intCookieValueEnd = header.IndexOf(";")

    cookie2.Path = header.Substring(intCookieNameEnd + 1, intCookieValueEnd - 1 - intCookieNameEnd)

    header = header.Remove(0, intCookieValueEnd + 1)

    'Getting whether cookie is secure

    If header.Trim.Equals("Secure") Then

    cookie2.Secure = True

    End If

     

    'Add cookies to cookie container.

    cookies.Add(cookie1)

    cookies.Add(cookie2)

    myHttpWebRequest = Nothing

    myHttpWebResponse = Nothing

    Return cookies

     

    hopefully this helps.

    Chris McNear

     

    Monday, September 11, 2006 8:22 PM
  • I can't find the problem , but I think the guys from http://www.vbulletin.com/ made those forums as safe as possible. I can't track that hash code. I used the same cookies from one request to another with no success. I even tried to copy all the gets and the posts generated by IExplorer. Something is missing and I think that sessionHas is.

    If you find any solution you can try it on any vbulletin forum ( for example http://forum.pixelrage.ro/ ) .

    Thanks for your patience

    Monday, September 11, 2006 11:32 PM