Get started for free
Visual Studio Developer Center >
> 
> 
StreamReader.ReadToEnd() is reading way past the "end"

Answered StreamReader.ReadToEnd() is reading way past the "end"

  • Friday, April 13, 2012 10:17 PM
     
     
    Sign In to Vote
    0

    I am getting this, which has been reported many others:

    A potentially dangerous Request.Form value was detected from the client (txtAnswer="...ceived

    Here is my code:

    HttpWebResponse WebResp = (HttpWebResponse)request.GetResponse();
                    Stream Answer = WebResp.GetResponseStream();
                    StreamReader _Answer = newStreamReader(Answer);
                    txtAnswer.Text = _Answer.ReadToEnd();

    The answer in the txtAnswer is what is expected: "Message Received". That was the exact message sent.  But when I look at the value in _Answer, here is what I see: (Likely the reason for the error I am seeing)

    "Message Received\r\n\r\n<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Transitional//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd\">\r\n\r\n<html xmlns=\"http://www.w3.org/1999/xhtml\">\r\n<head><title>\r\n\r\n</title></head>\r\n<body>\r\n    <form method=\"post\" action=\"SourceHandler.aspx\" id=\"form1\">\r\n<input type=\"hidden\" name=\"__VIEWSTATE\" id=\"__VIEWSTATE\" value=\"/wEPDwULLTE2MTY2ODcyMjlkZGotW3kgAvjq6iPkdL/rVgR46eAKUmHfwgy6QZiJlMaC\" />\r\n\r\n    <div>\r\n    \r\n    </div>\r\n    </form>\r\n</body>\r\n</html>\r\n"

    If I change the code to : txtAnswer.Text = _Answer.ReadLine(), what I get is "Message Received" and there is no error.

    If I change the code to:

    string result;
    While ((result = _Answer.ReadLine()) != null)
    {
         txtAnswer.Text += result;
    }
    Again I get all of the gibberish displayed in the textbox, and the error code.

    Would appreciate some guidance.  The question about this error has been asked on the internet since 2008.  Everyone answers by telling people to turn off the validation.  That does not sound right.

    Would appreciate some enlightenment. Thanks
    Javed

     

All Replies

  • Saturday, April 14, 2012 2:54 PM
     
     Answered Has Code
    Sign In to Vote
    1

    I spent a few hours looking into this request.  I'm not usre if my answers are going to work since you didn't include a webpage.  I used one of my webpages and ran into a few problems.

    I think your problem may just be the encoding so try this first

    From :

    Stream Answer = WebResp.GetResponseStream();

    StreamReader(Answer);

    To :

    StreamReader _Answer = StreamReader(WebResp.GetResponseStream(), System.Text.

    I was getting an error that no length was specified and found the only way of solving this problems was adding a callback routine.  I used the webpage below as reference.  The webpage solution didn't work for me and I had to make some changes to get proper results.  YOu may not have the same problems I did with your webpage.

    http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.begingetrequeststream.aspx

    using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.IO;
namespace Webpage
{
    public class RequestState
    {
        // This class stores the request state of the request.
        public WebRequest request;
        public RequestState()
        {
            request = null;
        }
    }
    class Program
    {
        public static void Main(string[] args)
        {
            string URL = @"http://my webpage/";
            WebRequest myWebRequest =
               (WebRequest)WebRequest.Create(URL);
            myWebRequest.Method = "POST";
            // Create an instance of the RequestState and assign 
            // 'myWebRequest' to it's request field.    
            RequestState myRequestState = new RequestState();
            myRequestState.request = myWebRequest;
            myWebRequest.ContentType = "text/html; charset=UTF-8";  
           
            HttpWebResponse WebResp;
            // Start the Asynchronous 'BeginGetRequestStream' method call.    
            IAsyncResult r = (IAsyncResult)myWebRequest.BeginGetRequestStream(
                new AsyncCallback(ReadCallback), myRequestState);
            WebResp = (HttpWebResponse)myWebRequest.GetResponse();
            StreamReader _Answer = new StreamReader(WebResp.GetResponseStream(), System.Text.Encoding.UTF8);
            string Text = _Answer.ReadToEnd();
            
        }
        private static void ReadCallback(IAsyncResult asynchronousResult)
        {
            RequestState myRequestState = (RequestState)asynchronousResult.AsyncState;
            WebRequest myWebRequest = myRequestState.request;
            // End the Asynchronus request.
            Stream streamResponse = myWebRequest.EndGetRequestStream(asynchronousResult);
            streamResponse.Close();
           
        }
    }
}

    jdweng

     
  • Sunday, April 15, 2012 2:12 AM
     
      Has Code
    Sign In to Vote
    0

    Joel,

    Thank you for your response.

    To demonstrate the issue, I have created a minimal set of projects (actually added a TestPage to an existing Web project to make the request and a minimal Web Project to send back the response)

    Here is TestPage.aspx, it is largely blank page with a button, to send the request, and a TestBox to display the response. Here is the entire code.

    using System;
using System.Net;
using System.Collections.Generic;
using System.Linq;
using System.IO;
 
namespace HelloTest
{
    public partial class TestPage : System.Web.UI.Page
    {
        protected void Page_Load(object sender, EventArgs e)
        {
        }
 
        protected void btnSend_Click(object sender, EventArgs e)
        {
            HttpWebRequest request = WebRequest.Create(new Uri("http://localhost:1229/Default.aspx", UriKind.Absolute)) as HttpWebRequest;
            request.Method = "POST";
            request.ContentType = "text/xml";
            string message = "Good Morning";
            byte[] byteArray = System.Text.Encoding.UTF8.GetBytes(message);
            Stream PostData = request.GetRequestStream();
            PostData.Write(byteArray, 0, message.Length);
            PostData.Close();
            HttpWebResponse WebResp = (HttpWebResponse)request.GetResponse();
            StreamReader _Answer = new StreamReader(WebResp.GetResponseStream());
            txtAnswer.Text = _Answer.ReadToEnd();
         }
    }
}

    The answering Website has only a single page. I created it in VS2008 and then converted to VS2010 - Net Framework 4.0

    The Default page is blank. Code behind:

    using System;
using System.Collections.Generic;
using System.Web;
using System.IO;
 
namespace WebApplication1
{
    public partial class _Default : System.Web.UI.Page
    {
        protected void Page_Load(object sender, EventArgs e)
        {
            HttpRequest req = Request;
            if (req.InputStream.Length > 0)
            {
                StreamReader reader = new StreamReader(req.InputStream);
                string inComing = reader.ReadToEnd();
                string result = "Good Afternoon";
                Response.Write(HttpUtility.HtmlEncode(result));
            }
        }
    }
}

    I expect to see the reponse "Good Afternoon". Here is what I have in the TextBox:

    Good Afternoon<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "<html">http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml" ><head><title> Untitled Page</title></head><body>    <form name="form1" method="post" action="Default.aspx" id="form1"><input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="/wEPDwUJNzgzNDMwNTMzZGTPhInrIHqnQxhk9H2lxlK9rOkPyniuEfUyXkSgg39Hkw==" />    <div>        </div>    </form></body></html>

    May be I am doing something wrong. It all started with the following error:

    A potentially dangerous Request.Form value was detected from the client

    When I searched the internet there were a truckload of hits with no clear cut resolution. Thanks for your help.

    Javed


    PS: I have tried using : 
    StreamReader _Answer = new StreamReader(WebResp.GetResponseStream(), System.Text.Encoding.UTF8);
    but the result is the same.
    • Edited by Jav Sunday, April 15, 2012 1:51 PM
    •  
     
  • Sunday, April 15, 2012 2:00 PM
     
     
    Sign In to Vote
    0

    The following from

             http://msdn.microsoft.com/en-us/library/system.io.streamreader.readtoend(v=VS.71).aspx

    might be the answer:

    "ReadToEnd works best when you need to read all the input from the current position to the end of the stream. If more control is needed over how many characters are read from the stream, use Read(Char[], Int32, Int32), which generally results in better performance.

    ReadToEnd assumes that the stream knows when it has reached an end. For interactive protocols, in which the server sends data only when you ask for it and does not close the connection, ReadToEnd might block indefinitely and should be avoided."

    I need to change my code.

    Javed

     
  • Sunday, April 15, 2012 9:00 PM
     
      Has Code
    Sign In to Vote
    1

    What you are sending is wrong, not what you are recieving.  You are using HTTP services which means you are getting HTML data, but all you want to sent is data.  You are seeing all the HTML tags.  You message that is being sent should be wrapped in a tag, it shouldn't be at the beginning of the data stream.

    take the data below a sotre it in a file on your computer call translate.html.  then double click on the file so it opens in your webbrowser.  The two text boxes in the  file has "id=" which identifies the boxes.  I can extract the data from the boxes by by putting the text into a IHTMLDocument.  See code below

    StreamReader reader = new StreamReader(req.InputStream);
string inComing = reader.ReadToEnd();
//micorosft HTML document library
mshtml.HTMLDocument doc = new mshtml.HTMLDocument();
doc.write(inComing);
string results = doc.getElementsByTagName("box1").toString();
or
string results = doc.getElementById("box1").toString();

    Create file with data below

    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Google Translate API</title>
</head>
<body>
<form name="f" id="f" action="#" onsubmit="translate(); return false;">
<textarea name="foreign_text" id="foreign_text" rows="4"
cols="60"></textarea>
<br />
<br />
<input type="submit" id="submit_button" value="Translate into English"
onfocus="this.blur();" />
<br />
<br />
<textarea name="translation" id="translation" rows="4" cols="60"
onfocus="this.select();" readonly="true"></textarea></form>
<br />
<script type="text/javascript" src="http://www.google.com/jsapi"></script>
<script type="text/javascript">
google.load("language", "1");
function translate() {var
originaltext=document.forms["f"].foreign_text.value;
google.language.translate(originaltext, "", "en", function(result) {
document.forms["f"].translation.value = (result.error)?("Error:
"+result.error.message):result.translation; }); }
</script>
</body>
</html>

    jdweng

     
  • Monday, April 16, 2012 3:51 AM
     
     
    Sign In to Vote
    0

    Joel,

    Thank you.  Your help is much appreciated.

    If you are talking about the following line in my Response:
            Response.Write(HttpUtility.HtmlEncode(result));
    I was trying everything I could think of see if it would help, that one just happened to be the last.  I have the rame result with:
            Response.Write(result);

    I am now using:    txtAnswer.Text = _Answer.ReadLine();  and it is working fine.  All responses I will be getting in this situation should be no more than a line.

    I am also looking at the Asynch code that you provided in your first post.  I had to use the Asynch method when I was trying to send the Request from Silverlight. But I ran into the problem of Cross Domain issues when I tried to get at my Request data from the CallBack method.  I had to mess with the Dispatcher and SyncInvoke.  I like your technique of using a class to hold the data - it is much neater.

    I do have a question though.  Shouldn't the  WebRequest.GetResponse() code be in a separate callback method?  If you agree, where would you make the call to that callback method?

    I really appreciate your help.  Thanks

    Javed

     
  • Monday, April 16, 2012 9:06 AM
     
     
    Sign In to Vote
    1
    Do you need a callback method for WebRequest.GetResponse()?  There is a built in callback in the base class.  You only need to write your own when the base class doesn't do everything you want.  The request is a blocking process that won't return until the operation is completed.  the request only send out an asynchronous message.  The only thing that can fail is if the socket is closed.  You also would get an error if the URL didn't exist (or was down).  I'm not sure if the callback would get that failure message.

    jdweng

     
  • Monday, April 16, 2012 11:14 AM
     
     
    Sign In to Vote
    0

    I didn't know that.  Thanks.  Does that apply also if your Request is for a file download?

    Javed

     
  • Monday, April 16, 2012 12:15 PM
     
     Answered
    Sign In to Vote
    0

    You have to learn to read inbetween the lines when you read any Microsoft Documentation.  Don't assume anything.  Read all the documentation.  You may just have to try a bad file name and see what is returned then write code to handle you exception.  A lot of the documenation isn't any documentation but just say "This is an Example".  I'm just as frustrated as the rest of the world with microsoft documenation.  I can go on for hours with the things that are wrong with Microsoft Documentation. 

    For software to be certified for use on windows it has to meet 8th grade reading level for docuemtnation. ?Why only 8th grade?  8th grade limits the following

    a) The number of paragraphs per page

    b) The number of sentances per paragraph

    c) The number of words per sentence

    d) The number of syllables per word

    e) The list of words that can be used.

    All classes behave similar when overriden and most of the Net libraries are classes.  For example see this webpage

    http://msdn.microsoft.com/en-us/library/system.net.webrequest.getresponse(v=vs.90).aspx

    It says the following

    1) When overridden in a descendant class, returns a response to an Internet request.

    Note: It says "When overridden".  If it is not overriden then the base class does the function

    2) Below is says what to do for synchronouws and asynchronous operation.  The default is synchronous where it returns any response.  It doesn't specify what happens when an error occurs.  So you have to read between the lines and look at the asynchronus methods to see if you can get the error conditions.

    The GetResponse method sends a request to an Internet resource and returns a WebResponse instance. If the request has already been initiated by a call to GetRequestStream, the GetResponse method completes the request and returns any response.

    The GetResponse method provides synchronous access to the WebResponse. For asynchronous access, use the BeginGetResponse and EndGetResponse methods.

    3) Now you go to the webpage for WebRequest.BeginGetRequestStream and you don't find a lot more information

    It says below "An object containing state information for this asynchronous request."

    What the hell does the state information tell you.  Does it tell you that the file is'nt found???

    callback
    Type: System.AsyncCallback
    The AsyncCallback delegate.
    state
    Type: System.Object
    An object containing state information for this asynchronous request.

    4) I finally stumbled on lots of exception classes that may help

    a) ArgumentException Class

    http://msdn.microsoft.com/en-us/library/system.argumentexception(v=vs.90).aspx

    b) ArgumentNullException Class

    http://msdn.microsoft.com/en-us/library/system.argumentnullexception(v=vs.90).aspx

    c) ArgumentOutOfRangeException Class

    http://msdn.microsoft.com/en-us/library/system.argumentoutofrangeexception(v=vs.90).aspx

    d) Exception class

    http://msdn.microsoft.com/en-us/library/system.exception(v=vs.90).aspx

    Note : you will see a lot more exception on the index on the left side of this webpage. I'm not sure which applies to a file not found on the internet.

    http://msdn.microsoft.com/en-us/library/system.asynccallback(v=vs.90).aspx

    jdweng

     
  • Monday, April 16, 2012 6:57 PM
     
     
    Sign In to Vote
    0

    I totally agree with you.  Unfortunately many of the books are no different.  I have 4 C# books, and often I have to look in all four to get an adequate answer. The other day I needed to look up the syntax for the UTC time.  The 3 C# 4.0 books do not even have UTC in the index.  One of those had a much fatter C# 3 edition which I happen to have - and this one has 8 entries for UTC in the index, and a multi-page coverage.

    Javed

     
  • Monday, April 16, 2012 7:13 PM
     
     
    Sign In to Vote
    1
    UTC comes from the Unix world.  Unix is built from C.  And C is built from only around 70 primitive C language functions (C is built from C).  To develop language on any PC, you simply write 70 macro functions.  The C compiler is written in using the primitive functions and then can compile the more advance C functions.  UTC functions are based on one of the primitives the reads the clock on the PC.

    jdweng

     
  • Tuesday, April 17, 2012 7:49 PM
     
     Answered
    Sign In to Vote
    0

    I kinda got lost in the thread, but the important thing is in your Page.

    When you want to just send the plain text part and ignore any of the actual Page data, you need to close the response early:

    Response.Clear();
    Response.Write("Just one line of text");
    Response.Flush();
    Response.Close(); or Application.CompleteRequest(); or Response.End()

    This is a resource heavy way of doing this. There are multiple blog posts about this subject on why you should not use this:

    • See also: http://stackoverflow.com/questions/1087777/is-response-end-considered-harmful/3917180#3917180
    • See also: https://blogs.msdn.com/b/aspnetue/archive/2010/05/25/response-end-response-close-and-how-customer-feedback-helps-us-improve-msdn-documentation.aspx?Redirected=true

    If you don't need all the extra features of the Page class, it's better to create a custom web handler. That allows you to stream out exactly what you want, without adding the overhead of building up control trees, parsing complex postbacks, viewstate and other things.

    See:

    • http://msdn.microsoft.com/en-us/library/bb398986.aspx
    • http://www.dotnetperls.com/ashx

    My blog: blog.jessehouwing.nl

     
  • Wednesday, April 18, 2012 6:31 AM
     
     
    Sign In to Vote
    0

    What Jesse Houwing says is completely true. The reason you encounter that error is that your target page contains normal HTML tages "<html><body> ..." alongside the returned output which is seemed to be sent to the page using Response.Write. You should either use an .asmx web service or a WCF service instead of a normal ASP.NET .aspx page, use Response.End() as Jesse says or at least clear out all the html tags in that target page (if that page is used merely as a service and is not intended to be requested normally by users).