locked
how to get the values from a web site that uses AJAX functionality RRS feed

  • Question

  • User-1039333075 posted

    Dear ASP.NET Developers,

    I want to web scrape some values from a web site. I had managed to do this by the code snippet given below, but the web site now uses AJAX functionality to display the values in a regular interval.

                string requestUrl = "http://www.buranburanburan.com/buranburan/buran.aspx";
                Uri address = new Uri(requestUrl);

                HttpWebRequest request = (HttpWebRequest) WebRequest.Create(address);
                HttpWebResponse response = (HttpWebResponse) request.GetResponse();
                StreamReader stream = new StreamReader(response.GetResponseStream());
                string source = stream.ReadToEnd();
                source = source.Substring(source.IndexOf("USDTRL", 0), 40);
                source = source.Substring(34, 6);

    The web site now uses AJAX rather than directly outputting the values:

            <script type="text/javascript">
            /*function SendQuoteTableAjax()
            {
                var dt = new Date();
                httpRequest.open( "GET", "./somewebpage.aspx?callajax=true&dt="+dt.getTime(),  true);
                httpRequest.send(null);
                setTimeout("SendTableQuoteAjax()",20000);
            }
            setTimeout("SendQuoteTableAjax()",20000);*/
            </script>

    How can I get the values now?

    Thanks in advance,

    buran

    Sunday, February 15, 2009 9:02 AM

Answers

  • User1250677129 posted

     AJAX is working using XMLHttpRequest Object. XMLHttpRequest equivalent object is not available in .net directly. So you have to create an interface and implement the methods supported by Javascript Xmlhttprequest.  As this is asyncronous, you need to specify a method to handle the data when the server is responsed.

     

    All the methods needed are already implemented and available in the below link.

    http://msdn.microsoft.com/en-us/magazine/bb985675.aspx

    Let me know if you have any issues.

     

    Thanks,

    Arunkumar

    MARK AS ANSWER IF IT HELPS YOU

    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Monday, February 16, 2009 7:39 AM

All replies

  • User1250677129 posted

    Please refer this for more info.

    http://msdn.microsoft.com/en-us/magazine/cc301587.aspx

     You have to implement asynchronous web client. It should be like XMLHTTPRequest in Client side.

    Thanks,

    Arunkumar

    Sunday, February 15, 2009 9:38 AM
  • User-1039333075 posted

    Thank you Arunkumar, but could you please explain this a little more?

    Monday, February 16, 2009 6:37 AM
  • User1250677129 posted

     AJAX is working using XMLHttpRequest Object. XMLHttpRequest equivalent object is not available in .net directly. So you have to create an interface and implement the methods supported by Javascript Xmlhttprequest.  As this is asyncronous, you need to specify a method to handle the data when the server is responsed.

     

    All the methods needed are already implemented and available in the below link.

    http://msdn.microsoft.com/en-us/magazine/bb985675.aspx

    Let me know if you have any issues.

     

    Thanks,

    Arunkumar

    MARK AS ANSWER IF IT HELPS YOU

    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Monday, February 16, 2009 7:39 AM