none
How to grab dynamic html data programmatically which load into browser control by ajax RRS feed

  • Question

  • couple of months back i load a site into web browser control. the site content / html was loading dynamically by ajax. so when i go to their site by chrome browser and try to see the html by view page then i did not find those text which was loading dynamically in browser.

    so i could not find text from browser control. i am looking for a way to grab those html content using browser control which is loading by ajax.

    please guide me with some sample code that how to grab dynamic html data programmatically which load into browser control by ajax.

    good browser collection for .net link https://stackoverflow.com/a/37352322/13722367

    thanks



    • Edited by Sudip_inn Friday, July 10, 2020 4:12 PM
    Thursday, July 9, 2020 3:09 PM

Answers

All replies

  • Hi Sudip_inn,

    Thank you for positng here.

    I found some links to get the data loaded by Ajax, please see if it's helpful.

    htmlagilitypack and dynamic content issue

    Get HTML in C# from page that Loads Dynamic Data

    Retrieve ajax/JavaScript return results from webpage in c#

    How to extract dynamic ajax content from a web page

    Best Regards,

    Timon


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Friday, July 10, 2020 2:12 AM
  • Thanks i will check and let you know.
    Friday, July 10, 2020 6:48 AM
  • I found one code sample from here https://stackoverflow.com/a/2540531/13722367

    which seems help me. here is that code

    using System;
    using System.Windows.Forms;
    
    namespace WebBrowserDemo
    {
        class Program
        {
            public const string TestUrl = "http://www.w3schools.com/Ajax/tryit_view.asp?filename=tryajax_first";
    
            [STAThread]
            static void Main(string[] args)
            {
                WebBrowser wb = new WebBrowser();
                wb.DocumentCompleted += new WebBrowserDocumentCompletedEventHandler(wb_DocumentCompleted);
                wb.Navigate(TestUrl);
    
                while (wb.ReadyState != WebBrowserReadyState.Complete)
                {
                    Application.DoEvents();
                }
    
                Console.WriteLine("\nPress any key to continue...");
                Console.ReadKey(true);
            }
    
            static void wb_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
            {
                WebBrowser wb = (WebBrowser)sender;
    
                HtmlElement document = wb.Document.GetElementsByTagName("html")[0];
                HtmlElement button = wb.Document.GetElementsByTagName("button")[0];
    
                Console.WriteLine(document.OuterHtml + "\n");
    
                button.InvokeMember("Click");
    
                Console.WriteLine(document.OuterHtml);           
            }
        }
    }

    Friday, July 10, 2020 6:51 AM
  • @Timon can you please share some web site address which load text data by ajax and those data can not be seen by view page source.

    please give me some url. thanks

    Friday, July 10, 2020 6:52 AM
  • Hi Sudip_inn,

    I really want to help you but I’m afraid I can’t. I’m not familiar with web technology.

    Maybe you can go to the ASP.Net forum and ask them to write a simple example for you, they should be familiar with it.

    Best Regards,

    Timon


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Friday, July 10, 2020 8:04 AM
  • sorry this is not web related question. i am using web browser control in winform where i load a web site which load data dynamically by ajax.

    thanks

    Friday, July 10, 2020 11:44 AM
  • This is how i tried. i load a web site into web browser control. the web site load more data when we scroll down.

    this web site load data dynamically by ajax. i try to read all dynamic h3 tag loaded by ajax but my code did not work.

    here is my code

    private void BrowserTest_Load(object sender, EventArgs e)
            {
                webBrowser1.Navigate("https://www.pinterest.com/pin/517210338432366716/");
            }
    
            private void webBrowser1_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
            {
                if (webBrowser1.ReadyState == WebBrowserReadyState.Complete)
                {
                    HtmlElement elm = webBrowser1.Document.GetElementById("h3"); // Get "abc" element by ID
                    //Console.WriteLine("elm.InnerHtml(DocumentCompleted):" + elm.InnerHtml);
                    if (elm != null)
                    {
                        elm.AttachEventHandler("onpropertychange", new EventHandler(handler));
                    }
                }
            }
    
            private void handler(Object sender, EventArgs e)
            {
                HtmlElement div = webBrowser1.Document.GetElementById("h3");
                if (div == null) return;
                String contentLoaded = div.InnerHtml;
            }
    
            private void btnScrollDown_Click(object sender, EventArgs e)
            {
                if (webBrowser1.Document != null)
                {
                    webBrowser1.Document.Window.ScrollTo(0, webBrowser1.Document.Body.ScrollRectangle.Height);
                }
            }

    Friday, July 10, 2020 6:53 PM