locked
Can anyone give me a specialized library about scrap dynamic website (js) in C# RRS feed

  • Question

  • User1045051390 posted

    I want a c#(or f#) library for scraping dynamic websites (Specifically YouTube). I found a few libraries like anglesharp/anglesharp.js(give an error for .net core 3.1), Html Agility Pack(not working dynamic website) and pseudo-Specialized library selenium(but this library for UI testing- I don't want to use this library).

    Does C # have a good library for this purpose?

    Friday, April 17, 2020 8:36 PM

Answers

  • User475983607 posted

    My goal is to extract channel videos with information of each (ex title description, video link, tags and so on)

    Then I recommend learning how to use the youTube REST API. https://developers.google.com/youtube/v3

    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Monday, April 20, 2020 3:30 PM

All replies

  • User303363814 posted

    What does "not working dynamic website" mean?  What problem do you have?

    Saturday, April 18, 2020 12:35 AM
  • User288213138 posted

    Hi aemanes,

    I want a c#(or f#) library for scraping dynamic websites (Specifically YouTube). I found a few libraries like anglesharp/anglesharp.js(give an error for .net core 3.1), Html Agility Pack(not working dynamic website) and pseudo-Specialized library selenium(but this library for UI testing- I don't want to use this library).

    If you need to scrape a website you can use ScrapySharp scraping framework. You can add it to a project as a nuget.

    https://www.nuget.org/packages/ScrapySharp/

    You can also try to use WebScraping Library. WebScraper provides a powerful framework to extract data and files from websites using C# code.

    https://ironsoftware.com/csharp/webscraper/

    Best regards,

    Sam

    Saturday, April 18, 2020 4:02 AM
  • User1045051390 posted

    IronWebscraper has a license , scrapySharp working on the dynamic website?

    Saturday, April 18, 2020 7:48 AM
  • User288213138 posted

    Hi armanes,

    IronWebscraper has a license , scrapySharp working on the dynamic website?

    Here some demo about how to Scrape Data from Websites in C#.

    https://ironsoftware.com/csharp/webscraper/tutorials/webscraping-in-c-sharp/#helpscout-support

    Best regards,

    Sam

    Monday, April 20, 2020 2:25 AM
  • User1045051390 posted

    Hi armanes,

    armanes

    IronWebscraper has a license , scrapySharp working on the dynamic website?

    Here some demo about how to Scrape Data from Websites in C#.

    https://ironsoftware.com/csharp/webscraper/tutorials/webscraping-in-c-sharp/#helpscout-support

    Best regards,

    Sam

    thank you for giving me information. I want scrap youtube Channels, can IronWebScraper click on elements or page scroll down (for extract all videos)?

    Monday, April 20, 2020 8:13 AM
  • User475983607 posted

    thank you for giving me information. I want scrap youtube Channels, can IronWebScraper click on elements or page scroll down (for extract all videos)?

    Why are you screen scraping? Why don't you simply create a developer account and take advantage if the youTube API?  What is your goal?

    Monday, April 20, 2020 11:11 AM
  • User1045051390 posted

    armanes

    thank you for giving me information. I want scrap youtube Channels, can IronWebScraper click on elements or page scroll down (for extract all videos)?

    Why are you screen scraping? Why don't you simply create a developer account and take advantage if the youTube API?  What is your goal?

    My goal is to extract channel videos with information of each (ex title description, video link, tags and so on)

    Monday, April 20, 2020 3:24 PM
  • User475983607 posted

    My goal is to extract channel videos with information of each (ex title description, video link, tags and so on)

    Then I recommend learning how to use the youTube REST API. https://developers.google.com/youtube/v3

    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Monday, April 20, 2020 3:30 PM
  • User1045051390 posted

    armanes

    My goal is to extract channel videos with information of each (ex title description, video link, tags and so on)

    Then I recommend learning how to use the youTube REST API. https://developers.google.com/youtube/v3

    Thank you very much

    Monday, April 20, 2020 3:47 PM