locked
Get Page Source with HttpClient RRS feed

  • Question

  • Is there a way to do it?  The "related topics" brings me to a page that says you need to use the WebClient control to get the actual page source, but there is no WebClient class available.  So, is it possible to get the actual page source through an app?

    If not possible with the HttpClient, is there a way to make sure they use the same connection?  I am currently using the HttpClient to log into the page, and I need to keep the connection alive to ensure I keep the same Session value.  Or should I go another route?

    Tuesday, September 11, 2012 3:49 AM

Answers

  • Thank you both for your help.  Ultimately, I couldn't get the socket method to work for me, but I did find a post (http://stackoverflow.com/questions/11660743/c-sharp-metro-httpclient-not-receiving-cookie-on-postasync) that lead me to my working solution.

    CookieContainer cookieJar = new CookieContainer();
    HttpClientHandler handler = new HttpClientHandler()
    {
    	CookieContainer = cookieJar
    };
    handler.UseCookies = true;
    handler.UseDefaultCredentials = false;
    HttpClient client = new HttpClient(handler as HttpMessageHandler)
    {
    	BaseAddress = uriGame
    };
    
    HttpResponseMessage response = await client.GetAsync(client.BaseAddress);
    Stream responseStream = await response.Content.ReadAsStreamAsync();
    using (StreamReader responseReader = new StreamReader(responseStream))
    {
    	//String sLine = responseReader.ReadLine();
    	String sResponse = responseReader.ReadToEnd();
    	String sWait = "wait";
    }
    
    HttpContent content = new FormUrlEncodedContent(new[]
    	{
    		new KeyValuePair<string, string>("mail", "emailaddress"),
    		new KeyValuePair<string, string>("password", "userpassword"),
    	});
    
    response = await client.PostAsync(uriLogin, content);
    responseStream = await response.Content.ReadAsStreamAsync();
    using (StreamReader responseReader = new StreamReader(responseStream))
    {
    	//String sLine = responseReader.ReadLine();
    	String sResponse = responseReader.ReadToEnd();
    	String sWait = "wait";
    }

    • Marked as answer by dgjohnson Friday, September 14, 2012 1:07 AM
    Friday, September 14, 2012 1:07 AM

All replies

  • actually can u elaborate it

    Try with this

    Uri myUri =new Uri("http://www.google.com");
    WebRequest myWebRequest= WebRequest.Create(myUri);
    WebResponse myWebResponse= myWebRequest.GetResponse();

    for keeping the connection alive i think u should go with socket programming...

    establish a connection

    write bytes to stream

    read response bytes

    again write so on.....

    Tuesday, September 11, 2012 8:49 AM
  • Right now it's something like:

    Dim httpSiteLogin As HttpClient
    Dim clientHandler As HttpClientHandler
    Dim response As HttpResponseMessage
    Dim streamResponse As Stream
    Dim streamResponseReader As StreamReader
    Dim sLine As String = ""
    
    clientHandler.Credentials = New NetworkCredential("username", "password")
    httpSiteLogin = New HttpClient(clientHandler)
    response = Await httpSiteLogin.GetAsync("http://www.website.com")
    response.EnsureSuccessCode()
    
    streamResponse = Await response.Content.ReadAsStreamAsync
    streamResponseReader = New StreamReader(streamResponse)
    Do While not streamResponseReader.EndOfStream
    	sLine = streamResponseReader.ReadLine
    	If sLine.IndexOf("SearchString") > 0 Then
    		sSessionId = sLine.Substring(IndexOf("SearchString") + 5, 6)
    		Exit Do
    	End If
    Loop
    
    ''' Make other calls to the site, using session id

    But, that is just parsing the response, not the actual page.  Is there a way to get the page source with HttpClient, or do I need to switch to WebRequest?  Would I then need to use the same WebRequest Object to ensure I'm in the same session?

    Also, do you have examples on how to use socket programming?  I'm not familiar with it.

    Tuesday, September 11, 2012 12:59 PM
  • Hi,

    As far as I know, you can use a WebView control to load a page and then use script to get the HTML in that page.

    Please follow this thread
    http://social.msdn.microsoft.com/Forums/en-US/winappswithnativecode/thread/8673b1dc-83d9-4e79-a95a-f3e000536b88

    Best regards,
    Jesse


    Jesse Jiang [MSFT]
    MSDN Community Support | Feedback to us

    Wednesday, September 12, 2012 8:55 AM
  • Perhaps, but I don't want to show the web page.  The point of the app is to get at the data without navigating to the web page.

    I have another thread about a slightly different issue for the same app, but have discovered that my real issue is that the HttpClient (and WebRequest, for that matter) are not using the same connection.  As a result, every time I try to navigate to the login screen, I get redirected to a welcome screen, which only shows up when there is no cookie for the site.  At least, I assume that's the issue, but when going to the web site through a normal browser, it only shows after I clear cookies.

    At any rate, it was suggested here that I use socket programming.  How would I go about doing that with the code I have posted?  I tried a little bit last night, but I was getting errors.  I'm sure I was simply using it incorrectly (I was trying to use the StreamWebSocket class), but couldn't find any proper examples.

    Wednesday, September 12, 2012 1:07 PM
  • I think socket programming using network stream will help you.. http://www.codeproject.com/Articles/10649/An-Introduction-to-Socket-Programming-in-NET-using http://socketprogramming.blogspot.in/
    Wednesday, September 12, 2012 6:45 PM
  • Thank you both for your help.  Ultimately, I couldn't get the socket method to work for me, but I did find a post (http://stackoverflow.com/questions/11660743/c-sharp-metro-httpclient-not-receiving-cookie-on-postasync) that lead me to my working solution.

    CookieContainer cookieJar = new CookieContainer();
    HttpClientHandler handler = new HttpClientHandler()
    {
    	CookieContainer = cookieJar
    };
    handler.UseCookies = true;
    handler.UseDefaultCredentials = false;
    HttpClient client = new HttpClient(handler as HttpMessageHandler)
    {
    	BaseAddress = uriGame
    };
    
    HttpResponseMessage response = await client.GetAsync(client.BaseAddress);
    Stream responseStream = await response.Content.ReadAsStreamAsync();
    using (StreamReader responseReader = new StreamReader(responseStream))
    {
    	//String sLine = responseReader.ReadLine();
    	String sResponse = responseReader.ReadToEnd();
    	String sWait = "wait";
    }
    
    HttpContent content = new FormUrlEncodedContent(new[]
    	{
    		new KeyValuePair<string, string>("mail", "emailaddress"),
    		new KeyValuePair<string, string>("password", "userpassword"),
    	});
    
    response = await client.PostAsync(uriLogin, content);
    responseStream = await response.Content.ReadAsStreamAsync();
    using (StreamReader responseReader = new StreamReader(responseStream))
    {
    	//String sLine = responseReader.ReadLine();
    	String sResponse = responseReader.ReadToEnd();
    	String sWait = "wait";
    }

    • Marked as answer by dgjohnson Friday, September 14, 2012 1:07 AM
    Friday, September 14, 2012 1:07 AM