locked
Is there a way to scrap data from websites using some api

    Question

  • I tried using htmlagilitypack but the api seems to have get changed and many properties and functions under that api is not working.

    for eg. string text = doc.DocumentNode.SelectSingleNode("/html[1]/body[1]/p[1]").InnerText;

    The SelectSingleNode is not working because of which I am unable to get data from document.

    Has anybody worked on this api? please help


    sushant sagar


    Friday, September 12, 2014 6:29 PM

Answers

All replies

  • Hi sushant,

    I'm having same scenario few years ago. what i did is  consume the webpage usinh HttpClient lbraray and then filter out relevent data with string filtering enables with default c#. It is very fine method and no issues with the exception and other factors.



    Hope that helps
    Please don't forget to up vote answers you like or which help you and mark one(s) which answer your question.

    ---------------------------------------

    M Prabath Maduranga Peiris
    Microsoft Student Partner
    Blogs : prabathsl.blogspot.com

    • Proposed as answer by Lahiru Nimantha Sunday, September 14, 2014 1:15 PM
    • Unproposed as answer by sushant sagar Sunday, September 14, 2014 1:54 PM
    Saturday, September 13, 2014 4:07 PM
  • Hi Sushant,

    For help with the HtmlAgilityPack you'll need to talk to its developers. They have a discussion page on their codeplex site at http://htmlagilitypack.codeplex.com/discussions .

    --Rob

    Saturday, September 13, 2014 5:37 PM
    Owner
  • in that case can you please provide me with some examples of code so that i can better understand.

    sushant sagar

    Saturday, September 13, 2014 8:24 PM
  • your xpath does not look correct to me. 

    That s why your code is not working. 

    HTMLagilitypack is quite powerful and easy to use.

    Sunday, September 14, 2014 12:54 AM
  • leave the xpath at least SelectSingleNode should work then i will think about the xpath.

    xpath is not that big a deal.


    sushant sagar

    Sunday, September 14, 2014 1:53 PM
  • Hi Suchant,

    I cannot provide sample code to help you use the HtmlAgilityPack.

    For help with the HtmlAgilityPack you'll need to talk to its developers. They have a discussion page on their codeplex site at http://htmlagilitypack.codeplex.com/discussions .

    --Rob

    Tuesday, September 16, 2014 1:17 AM
    Owner