locked
Protocol violation - can I change how sensible the test shall be to protocol errors?

    Question

  • I get this error when running a test on a web page that I have no possibility to change:

    RequestFailed: The server committed a protocol violation. Section=ResponseHeader Detail=CR must be followed by LF

    Is it possible to configure how the test shall handle this so the test will not fail?

    Tuesday, June 21, 2005 12:12 PM

Answers

  • Josh,

    Thanks so much.  I didn't realize the entry would work in a general application configuration file.  Adding the following to a default app.config worked:

    <system.net>
    <
    settings>
    <
    httpWebRequest useUnsafeHeaderParsing="true" />
    </
    settings>
    </
    system.net>

    I still don't like it, but thats the proof I needed for the manufacturer.  Thanks again!

    Friday, January 27, 2006 4:51 PM
  • Sten, this sounds like a bug so I'm going to need some more info to track it down.  What's the request like that's causing this error?  Are you using non-ASCII characters in a custom header value or in the URL?

    Josh
    Wednesday, June 22, 2005 1:36 PM
  • That code should not go in web.config.  It needs to be in vstesthost.exe.config which can be found in "c:\Program Files\Microsoft Visual Studio 8\Common7\ide\".  This error indicates a serious problem on your web server, though, so I'd certainly investigate what the source of the CR without LF is and fix it.

    Josh

    Thursday, January 12, 2006 1:41 PM
  • The config file entries need to be in vstesthost.exe.config if you are running locally, or in qtagent.exe.config if you are running the test from team test load agent. If you make these config file changes are you still seeing the problem?

    Ed.

    Thursday, January 12, 2006 1:45 PM
    Moderator
  • Reed,

    Adding the above-mentioned config file to any .NET app will cause it to use unsafe header parsing that should allow you to work with a web server that's not returning CRLFs.  The .NET framework's networking classes default to standards-based parsing, by default, for correctness and security reasons.

    Josh

    Friday, January 27, 2006 4:29 PM
  • As stated in this thread, the configuration entry needs to be in the application's config file, not web.config.  If your plugin is for winword.exe, try making the entry in winword.exe.config.

    Josh

    Monday, August 14, 2006 3:52 PM
  • It sounds like you are describing a proxy scenario...  I don't know what else "my pc is behind the linux server" could mean.

    I added the web reference you mention to a project and it worked just fine.  There was no error.

    I would say that you are behind an internet proxy server and the proxy server is reformatting the headers, causing the CRLF error.  Either remove the proxy server, or replace it with a standards compliant one.

    Tuesday, August 22, 2006 2:26 PM
  • That appears to be the case.

    This thread contains the code that changes the behavior of the CLR to allow unsafe headers.

    This thread also explains every possible code file that this code can go in - and when it is appropriate to use each file.

    We've also covered which application, in a multi-tier application scenario, is responsible for serving the invalid headers.

    Finally, it's been discovered that there may be intermediate servers translating valid headers into invalid ones.

    I'm not sure that anyone can help you further; it may be time to lock this thread and let it be...

    Wednesday, August 23, 2006 2:58 PM

All replies

  • Sten, this sounds like a bug so I'm going to need some more info to track it down.  What's the request like that's causing this error?  Are you using non-ASCII characters in a custom header value or in the URL?

    Josh
    Wednesday, June 22, 2005 1:36 PM
  • I have been trying to repeat this problem but the VS2005 enviroment crashes on clicking in the treeview of recorded requests and when replaying it. The request is one web page only.

    Another try after uninstalling and reinstalling the June CTP behaved the same way.

    The pages are generated by a content management system and I have no problem when using other tools like Mercury Loadrunner or OpenSTA.

    When VS2005 crashes it generates a error report that is sent to Microsoft and also the possibility to debug. Could the debug result in the form of dissasembled devenv.exe be of any interest for You?
    Wednesday, June 29, 2005 6:04 AM
  • I also encountered the same exeption:

    private
    void Form1_Load(object sender, EventArgs e)

    {

    string Url = "http://www.microsoft.com";

    HttpWebRequest Hppt = (HttpWebRequest) HttpWebRequest.Create(Url);

    Hppt.Timeout = 10000;

    HttpWebResponse WebResponse = (HttpWebResponse)Hppt.GetResponse();

    Encoding enc = Encoding.GetEncoding(1252);

    StreamReader ResponseStream = new StreamReader(WebResponse.GetResponseStream(), enc);

    string Html = ResponseStream.ReadToEnd();

    // Show response stream as html

    label1.Text = Html;

    WebResponse.Close();

    ResponseStream.Close();

    }


    The interesting thing is that for www.microsoft.com it works perfectly, but for other sites, such as www.freshdirect.com it raises the aforementioned exception.

    Just a wild guess, maybe it has to do with the differnce between unix and windows.

    I think WebClient doesn't raises th exception.

    Friday, July 29, 2005 4:02 PM
  • i had same error and fixed it by adding this to web.config...

    <configuration>
     <system.net>
     <settings>
     <httpWebRequest useUnsafeHeaderParsing="true" />
     </settings>
     </system.net>
     </configuration>


    hope that helps... :o)

    Friday, December 16, 2005 6:42 PM
  • I'm also getting same error while debugging the captured script.
    RequestFailed: The server committed a protocol violation. Section=ResponseHeader Detail=CR must be followed by LF

    This request is for login.  The response tab for this request is blank

    Please will you give me proper solution?

    Thanks,

    Thursday, January 12, 2006 6:00 AM
  • Hi Ichi,

    I have added the code into web.config file but yet getting same problem.  Anyone have solution?

    Thanks,

    Thursday, January 12, 2006 10:55 AM
  • That code should not go in web.config.  It needs to be in vstesthost.exe.config which can be found in "c:\Program Files\Microsoft Visual Studio 8\Common7\ide\".  This error indicates a serious problem on your web server, though, so I'd certainly investigate what the source of the CR without LF is and fix it.

    Josh

    Thursday, January 12, 2006 1:41 PM
  • The config file entries need to be in vstesthost.exe.config if you are running locally, or in qtagent.exe.config if you are running the test from team test load agent. If you make these config file changes are you still seeing the problem?

    Ed.

    Thursday, January 12, 2006 1:45 PM
    Moderator
  • I meet the same problem.I got an exception from System.Net.HttpWebRequest.GetResponse(), which says that there is protocol violation in SECTION=ResponseHeader, and Detail=CR must be followed by LF. I think that is a bug since IE can load the same web page.

    I've added the entires in vstesthost.exe.config but I can still see the problem.

    Wednesday, January 25, 2006 12:01 PM
  • Here is the problem...  It is not a bug...

    The Internet RFC 822, section 3.2, states that a Header Field in an internet message will be terminated with the character CRLF.  RFC 822 is referenced by nearly all other popular protocol RFCs, including RFC 2616 (the HTTP 1.1 RFC).  Unfortunately, RFC 2616 also has an antiquated recommendation under section 19.3 for "Tolerant Applications" that states "...the line terminator for message-header fields is the sequence CRLF.  However, we recommend that applications, when parsing such headers, recognize a single LF as a line terminator and ignore the leading CR.".  This recommendation is unfortunate because it is an obvious security risk to allow parsing of malformed headers and it has convinced some developers that there is no need to fix incomplete code that terminates a header with only LF or only CR.

    As for IE rendering a page with one of these malformed headers... IE is a notoriously sloppy browser.  You can build a page that breaks LOTS of RFC rules and IE will still render it.  This is what has lead to the number of security issues that IE has had.

    I've run into this error under VS05 using any of the WebResponse objects in a Windows Application.  There is no config file to be edited as mentioned in previous posts.  I'm trying to communicate with a hardware device that is embedded with windows mobile and provides a XML over HTTP host interface for communication.  Whatever webserver is running on this device formulates it's response headers using only a "LF" character.  Therefore none of the WebResponse objects can be created from a WebRequest object pointing to this device's URI.

    The manufacturer of the device tells me that I must contact Microsoft and show them that MS has designed the .NET Framework to not be tolerant of malformed headers.  I was hoping someone from the organization could reply to this post and verify that the CF expects fully compliant Internet Message Headers.  Or, if there is a work around at the framework level (that is, one that will work for a Windows Fat Client) to please post it.  Although, I still feel that allowing my app to parse unsafe headers is a security risk and it would be better to see someone from MS confirm that this is the case and is the reason why, out of the box, the framework doesn't allow unsafe headers.

    Friday, January 27, 2006 4:11 PM
  • Reed,

    Adding the above-mentioned config file to any .NET app will cause it to use unsafe header parsing that should allow you to work with a web server that's not returning CRLFs.  The .NET framework's networking classes default to standards-based parsing, by default, for correctness and security reasons.

    Josh

    Friday, January 27, 2006 4:29 PM
  • Josh,

    Thanks so much.  I didn't realize the entry would work in a general application configuration file.  Adding the following to a default app.config worked:

    <system.net>
    <
    settings>
    <
    httpWebRequest useUnsafeHeaderParsing="true" />
    </
    settings>
    </
    system.net>

    I still don't like it, but thats the proof I needed for the manufacturer.  Thanks again!

    Friday, January 27, 2006 4:51 PM
  • I meet the same problem.

    The Hello world could not be visited by vs2005 studio.

     

    Friday, March 10, 2006 3:31 AM
  • I meet the same problem :(

    I m building a project about mobile device application and I wanted to add web reference to tho my project. This problem occured.

    How can I solve this problem. There is no web config file on this project ?

    Thanks.

    Thursday, April 20, 2006 1:33 PM
  • The same setting can be applied to your app.config.

    Thursday, April 20, 2006 2:10 PM
  • Kemal -

    Be sure to read these threads completely, your answer is above your question.  But to clairify:

    A Windows Application project can be told to parse unsafe headers using an Application Configuration file.  Here is an example routine:

    1. In the Solution Explorer, right-click the Project Name
    2. Select Add, and then click New Item
    3. Select Application Configuration File, you can use the default name App.config, and click Open.  The following code is generated for you in the config file:
      • <?xml version="1.0" encoding="utf-8" ?>
        <
        configuration>
        </
        configuration>
    4. Now modify the pre-generated code to the following:
      • <?xml version="1.0" encoding="utf-8" ?>
        <
        configuration>
           <system.net>
              <settings>
                 <httpWebRequest useUnsafeHeaderParsing="true" />
              </settings>
           </system.net>
        </
        configuration>
    5. Save the modified application configuration file

    Your Windows Application will now properly parse unsafe headers.

    Thursday, April 20, 2006 2:56 PM
  • I saw this before but my problem is different from these. My project is mobile device application not windows application. I am working on VStudio.Net 2005. I dont see it with VS.net 2003.You can see screenshot from this link : http://rapidshare.de/files/18495742/error.JPG.html

     

    Thanks.

    Thursday, April 20, 2006 3:53 PM
  • This only works while running in the IDE.

     

    The problem still exists when you run the executable created during the build outside of the IDE (release or debug).

     

    This problem is a SERIOUS flaw in the design of the the .NET language.  Microsoft can not assume that every page on the net has a correct header.  It is the programmers responsibility to determine what to do.  This flaw has seriously pissed me off.  I have spent days coding an application for a client assuming that I had solved the problem with the addition to the config file only to find that I now cannot deliver the application.

     

    I need an IMMEDIATE work around or I am going to have to throw everything I have done away and write the application in another language/platform.

    Does anyone have a solution that will work after you build and run outside of the IDE?

     

     

    Friday, April 21, 2006 2:10 AM
  • Greg,

    The System.Net classes are following the RFC standard.  They support the config setting above to relax the header parsing rules for non-standard servers.

    Let's say your app is named xyz.exe, you have an xyz.exe.config with the above configuration in it and the headers are still not being parsed?  What exception are you getting?

    Josh

    Friday, April 21, 2006 2:15 PM
  • Greg,

    I just want to reinforce what Josh has stated.  If the config files exists with the compiled assembly than the problem is resolved in the released executable.

    I have successfully created an application that uses the config file posted above.  There must be a problem with your build...  As Josh stated, posting a detailed exception may help us help you.

    Also (again reinforcing what Josh stated) this behavior is not a flaw, bug, or error.  It is simply good programming that follows the rules for security reasons.  Internet RFCs exist for a reason, and they should be followed by any engineer that develops Internet enabled applications.

    Friday, April 21, 2006 3:50 PM
  • Kemal -

    Using a mobile device application doesn't really change anything...

    I realize that when you select Add New Item the "Application Configuration File" template is not displayed.  But don't worry about that.  Choose "Code File", name the new item "app.config", and then past the basic config data into the file:

    <?xml version="1.0" encoding="utf-8" ?>
      <
    configuration>
        <
    system.net>
          <
    settings>
            <
    httpWebRequest useUnsafeHeaderParsing="true" />
          </
    settings>
        </
    system.net>
      </
    configuration>

    Your Windows Mobile 2005 application will now parse the unsafe headers.

    Friday, April 21, 2006 4:14 PM
  •  

    I have found why my executable was not working even with the config settings.  It was because I was executing out of the obj and not bin directories.

    However, in response to rkimble and the entire Microsoft team that made the decision to make the default setting to block sites with bad headers:

    GetResponse call brings back static text, no activex or javascript execution occurs.  Therefore, there is not threat of malicious code executing from this call.  So were is the danger?

    So, in my opinion, this is a flaw.  This is Microsoft dictating which sites can be viewed or retreived and forcing all site owners to follow their standards.  I am currently retrieving many pages that have what Microsoft would consider bad headers with no problems.  The pages display in Internet Explorer.  Why is it that they are allowed to bring back these pages, but insist that we can't?  I would think that with my almost 20 years of programming experience that I should be capable of handling these situations the way that I need them to be handled in order to deliver a quality product to my customer.

    I sorry if this sounds hostile, it is not meant to be.  I am just stating my opinion supported by some facts.

    G McCallum

     

    Friday, April 21, 2006 4:16 PM
  • Not to fuel a big debate, as this is not really the place for it, but to be sure other readers understand what is actually happening:

    The parsing of message headers can lead to a buffer overflow if those headers are specifically malformed.  Buffer overflows are one of the top exploits that malicious Internet application developers target.  So parsing unsafe headers could indeed lead to problems in applications.

    Also understand that the Request For Comments (RFC) documents that define all Internet messages were not written by Microsoft.  The documents are maintianed mostly by IANA, and many that define core IP specifications were written by the Internet Engineering Task Force (IETF) and the Internet Engineering Steering Group (IESG).

    So Microsoft did not make they rules - they have only choosen to abide by them, as any good software developer should.

    Friday, April 21, 2006 4:27 PM
  • I changed the app conf file but this error has happened again. This is the secreen picture of the error. I couldnt solve it. Please help ?http://rapidshare.de/files/19775848/error.JPG.html

     

     

    Saturday, May 06, 2006 4:23 PM
  • Your project is a Web Service, not a Windows Application.  Try putting the configuration section in your Web.Config file instead of trying to use an App.Config file.
    Sunday, May 07, 2006 4:57 PM
  • Mr. rkimble

    I tried that you have told me.but  The same error happened again. This the picture of new  one.I want to add web reference with wizard to my MobileDeviceApplication. Is this clear enough ? Thanks for response.

     http://rapidshare.de/files/19961245/error2.JPG.html

     

    Monday, May 08, 2006 8:07 PM
  • I couldnt solve the problem. Anyone help me please?
    Sunday, May 21, 2006 11:20 AM
  • Did you ever figure this out?  Do you realize that it is the Web Service project that needs the configuration entries, not your mobile application?  The Web Service is generating the error so it must be in the Web Service project that you need to add the config info.

     

    Tuesday, June 20, 2006 8:13 PM
  • Hi
    Sorry for my English ! I'm French !
    I've devel a plug-in for office. And this plug-in is reading a SharePoint WebService !!
    I've try your solution, but I don't know in which file I must paste the code you give us !
    I have try every web.config file on my server, but none resolved my problem ! And I try to put it in the app.config of the plug-in, but I'm not sure it take it !!
    Any idea ?
     
    Many thx
    Sunday, August 06, 2006 7:29 AM
  • As stated in this thread, the configuration entry needs to be in the application's config file, not web.config.  If your plugin is for winword.exe, try making the entry in winword.exe.config.

    Josh

    Monday, August 14, 2006 3:52 PM
  • I couldnt solve the problem. I put the code into app.config file and then I build the mobile application project succesfuly!. I tried to add web reference and I met the same error: The server committed a protocol violation. Section=ResponseHeader Detail=CR must be followed by LF.  What is this ?  Is this a visual studio 2005 Bug ?  when I try in vs.net 2003 enviroment, this problem is not occured. I want to read an article about this issue. Anyone navigate me ? 

    Thanks.

    Wednesday, August 16, 2006 5:35 AM
  • Hi
    I don't understand where I can find the file winword.exe.config !! In my VS .Net 2005 Plugin I have only a app.config and this is the only .config file !!
    Is it this file ????????

    Many thx

    Cédric Petter

    -- EDIT --

    I've found !!
    I must write the code in the machine.config of the machine and use the app.config file !!!
    and the app.config must be an embedded resource ! and copied to the output dir !!!!
    Good bye

    Wednesday, August 16, 2006 6:17 AM
  • My Pc is behind the linux server. I dont have any problems when using direct internet connection. I tried it but I couldnt solve it yet. In this project there is a web service added before.When I click the Right mouse and Update Web Reference link, this error occured.  http://static.flickr.com/71/221926562_8125f2e063_o.jpg

     

     

     

    Tuesday, August 22, 2006 11:53 AM
  • It sounds like you are describing a proxy scenario...  I don't know what else "my pc is behind the linux server" could mean.

    I added the web reference you mention to a project and it worked just fine.  There was no error.

    I would say that you are behind an internet proxy server and the proxy server is reformatting the headers, causing the CRLF error.  Either remove the proxy server, or replace it with a standards compliant one.

    Tuesday, August 22, 2006 2:26 PM
  • Hi,

    My company uses a linux server as a gateway. There is a distribution "Censornet" downloaded from www.censornet.com. There is a proxy server on it but I do not know the details. i.e. I am behind a proxy sever that I can not change and configure. I am using a windows xp sp2 pc. I am runnig VS2005 on it. 

    when I want to add web reference to my project, I have the error message like kemal has.

    If the CR LF problem is because of the proxy server, I can tell it to the server admin. I read all of the posts under this topic, I added the code you describe but my problem is still going on.

    what can I do now?

    Tuesday, August 22, 2006 3:25 PM
  • hi rkimble. I think you cant help us anymore.Am I wrong ?
    Wednesday, August 23, 2006 2:42 PM
  • That appears to be the case.

    This thread contains the code that changes the behavior of the CLR to allow unsafe headers.

    This thread also explains every possible code file that this code can go in - and when it is appropriate to use each file.

    We've also covered which application, in a multi-tier application scenario, is responsible for serving the invalid headers.

    Finally, it's been discovered that there may be intermediate servers translating valid headers into invalid ones.

    I'm not sure that anyone can help you further; it may be time to lock this thread and let it be...

    Wednesday, August 23, 2006 2:58 PM