locked
How to extract data from another website thru WebAPI RRS feed

  • Question

  • User1242168447 posted

    Hi, I'm most of the time working in C# WinForm and ASP.Net MVC non-WebAPI...  but now I'd like to accomplish a WebAPI task given to me.

    I'm actually new to Web API, I practiced for few weeks video like this one and read up few books to understand it; I also have a solid background in .Net network programming in HttpWebquest/HttpClient but never known that WebAPI also use it until I read that in webapi book weeks ago.

    Please I need help to achieve something like the below :

    Example

    Both Website A & B are developed in ASP.Net MVC, and I am the developer for website A.

    Website A (www.mysite.com)

    Website B ( https://www.cars.com/research/suv/ )

    website A needs to extract data from specific page of website B thru WebAPI.

    Let assume the page to extract data from website B is (https://www.cars.com/research/suv/), and I need to extract everything from that page (image + title + description + text + details for each items[vehicle] )...

    Developers for website B gave me a list of API endpoints + ApiSecretKey and api direct link...   and told me that all requests should be called via GET ;

    and data extracted from website B, should be placed in website A in (www.mysite.com/pageVehicule).

    Please how do I accomplish it?

    Friday, October 11, 2019 8:10 AM

Answers

  • User61956409 posted

    Hi LetMeCode,

    LetMeCode

    Let assume the page to extract data from website B is (https://www.cars.com/research/suv/), and I need to extract everything from that page (image + title + description + text + details for each items[vehicle] )

    If you'd like to make request to website B, and grab and parse html data in C#. You can use some third-party libraries, such as HtmlAgilityPack, to extract data from Html document.

    https://www.nuget.org/packages/HtmlAgilityPack/

    LetMeCode

    Developers for website B gave me a list of API endpoints + ApiSecretKey and api direct link...   and told me that all requests should be called via GET

    If website B has exposed API endpoints for extracting data (or information) of vehicle, you can make request(s) and extract expected data from response directly rather than request to website B and parse html data.

    With Regards,

    Fei Han

    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Monday, October 14, 2019 2:08 AM

All replies

  • User753101303 posted

    Hi,

    I would suggest to use https://docs.microsoft.com/en-us/aspnet/web-api/overview/advanced/calling-a-web-api-from-a-net-client#send-a-get-request-to-retrieve-a-resource (HttpClient) to create a sample console app one step at a time and try the APIs they gave you.

    Then ask for help if you run into something on which you need help.

    Friday, October 11, 2019 8:33 AM
  • User61956409 posted

    Hi LetMeCode,

    LetMeCode

    Let assume the page to extract data from website B is (https://www.cars.com/research/suv/), and I need to extract everything from that page (image + title + description + text + details for each items[vehicle] )

    If you'd like to make request to website B, and grab and parse html data in C#. You can use some third-party libraries, such as HtmlAgilityPack, to extract data from Html document.

    https://www.nuget.org/packages/HtmlAgilityPack/

    LetMeCode

    Developers for website B gave me a list of API endpoints + ApiSecretKey and api direct link...   and told me that all requests should be called via GET

    If website B has exposed API endpoints for extracting data (or information) of vehicle, you can make request(s) and extract expected data from response directly rather than request to website B and parse html data.

    With Regards,

    Fei Han

    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Monday, October 14, 2019 2:08 AM