How to use get the response content html and save it to text file? RRS feed

  • Question

  • User-1050862601 posted


    I'm learning how to use asp to develop a web crawler. So I need to get the repsonse html and save it to text file. Then I can analyze the html based on the saved data. Does anyone have idea about how to achieve this? Thanks in advance.

    Thursday, January 11, 2018 5:02 PM


  • User-460007017 posted

    Hi suchanewbee,

    This link provide a sample for the web crawler:


    In addition, I could provide the asp.net application sample:

     string targetpath = @"d:\webcrawler\test.txt";
                Console.WriteLine("Please enter the target URL:");
                string URI = Console.ReadLine();
                WebRequest myrequest = WebRequest.Create(URI);
                WebResponse myresponse=  myrequest.GetResponse();
                Stream datastream = myresponse.GetResponseStream();
                StreamReader reader = new StreamReader(datastream, Encoding.Default);
                string htmlcontent = reader.ReadToEnd();

    Best Regards,

    Yuk Ding

    • Marked as answer by Anonymous Tuesday, September 28, 2021 12:00 AM
    Friday, January 12, 2018 6:16 AM