Protocol violation - can I change how sensible the test shall be to protocol errors?

Question

I get this error when running a test on a web page that I have no possibility to change:

RequestFailed: The server committed a protocol violation. Section=ResponseHeader Detail=CR must be followed by LF

Is it possible to configure how the test shall handle this so the test will not fail?

Answer 1

Sten, this sounds like a bug so I'm going to need some more info to track it down.  What's the request like that's causing this error?  Are you using non-ASCII characters in a custom header value or in the URL?

Josh

Answer 2

I have been trying to repeat this problem but the VS2005 enviroment crashes on clicking in the treeview of recorded requests and when replaying it. The request is one web page only.

Another try after uninstalling and reinstalling the June CTP behaved the same way.

The pages are generated by a content management system and I have no problem when using other tools like Mercury Loadrunner or OpenSTA.

When VS2005 crashes it generates a error report that is sent to Microsoft and also the possibility to debug. Could the debug result in the form of dissasembled devenv.exe be of any interest for You?

Answer 3

I also encountered the same exeption:

private void Form1_Load(object sender, EventArgs e)

{

string Url = "http://www.microsoft.com";

HttpWebRequest Hppt = (HttpWebRequest) HttpWebRequest.Create(Url);

Hppt.Timeout = 10000;

HttpWebResponse WebResponse = (HttpWebResponse)Hppt.GetResponse();

Encoding enc = Encoding.GetEncoding(1252);

StreamReader ResponseStream = new StreamReader(WebResponse.GetResponseStream(), enc);

string Html = ResponseStream.ReadToEnd();

// Show response stream as html

label1.Text = Html;

WebResponse.Close();

ResponseStream.Close();

}


The interesting thing is that for www.microsoft.com it works perfectly, but for other sites, such as www.freshdirect.com it raises the aforementioned exception.

Just a wild guess, maybe it has to do with the differnce between unix and windows.

I think WebClient doesn't raises th exception.

Answer 4

i had same error and fixed it by adding this to web.config...

<configuration>
 <system.net>
 <settings>
 <httpWebRequest useUnsafeHeaderParsing="true" />
 </settings>
 </system.net>
 </configuration>


hope that helps... :o)

Answer 5

I'm also getting same error while debugging the captured script.
RequestFailed: The server committed a protocol violation. Section=ResponseHeader Detail=CR must be followed by LF

This request is for login.  The response tab for this request is blank

Please will you give me proper solution?

Thanks,

Answer 6

Hi Ichi,

I have added the code into web.config file but yet getting same problem.  Anyone have solution?

Thanks,

Answer 7

That code should not go in web.config.  It needs to be in vstesthost.exe.config which can be found in "c:\Program Files\Microsoft Visual Studio 8\Common7\ide\".  This error indicates a serious problem on your web server, though, so I'd certainly investigate what the source of the CR without LF is and fix it.

Josh

Answer 8

The config file entries need to be in vstesthost.exe.config if you are running locally, or in qtagent.exe.config if you are running the test from team test load agent. If you make these config file changes are you still seeing the problem?

Ed.

Answer 9

I meet the same problem.I got an exception from System.Net.HttpWebRequest.GetResponse(), which says that there is protocol violation in SECTION=ResponseHeader, and Detail=CR must be followed by LF. I think that is a bug since IE can load the same web page.

I've added the entires in vstesthost.exe.config but I can still see the problem.

Answer 10

Here is the problem...  It is not a bug...

The Internet RFC 822, section 3.2, states that a Header Field in an internet message will be terminated with the character CRLF.  RFC 822 is referenced by nearly all other popular protocol RFCs, including RFC 2616 (the HTTP 1.1 RFC).  Unfortunately, RFC 2616 also has an antiquated recommendation under section 19.3 for "Tolerant Applications" that states "...the line terminator for message-header fields is the sequence CRLF.  However, we recommend that applications, when parsing such headers, recognize a single LF as a line terminator and ignore the leading CR.".  This recommendation is unfortunate because it is an obvious security risk to allow parsing of malformed headers and it has convinced some developers that there is no need to fix incomplete code that terminates a header with only LF or only CR.

As for IE rendering a page with one of these malformed headers... IE is a notoriously sloppy browser.  You can build a page that breaks LOTS of RFC rules and IE will still render it.  This is what has lead to the number of security issues that IE has had.

I've run into this error under VS05 using any of the WebResponse objects in a Windows Application.  There is no config file to be edited as mentioned in previous posts.  I'm trying to communicate with a hardware device that is embedded with windows mobile and provides a XML over HTTP host interface for communication.  Whatever webserver is running on this device formulates it's response headers using only a "LF" character.  Therefore none of the WebResponse objects can be created from a WebRequest object pointing to this device's URI.

The manufacturer of the device tells me that I must contact Microsoft and show them that MS has designed the .NET Framework to not be tolerant of malformed headers.  I was hoping someone from the organization could reply to this post and verify that the CF expects fully compliant Internet Message Headers.  Or, if there is a work around at the framework level (that is, one that will work for a Windows Fat Client) to please post it.  Although, I still feel that allowing my app to parse unsafe headers is a security risk and it would be better to see someone from MS confirm that this is the case and is the reason why, out of the box, the framework doesn't allow unsafe headers.

Answer 11

Reed,

Adding the above-mentioned config file to any .NET app will cause it to use unsafe header parsing that should allow you to work with a web server that's not returning CRLFs.  The .NET framework's networking classes default to standards-based parsing, by default, for correctness and security reasons.

Josh

Answer 12

Josh,

Thanks so much.  I didn't realize the entry would work in a general application configuration file.  Adding the following to a default app.config worked:

<system.net>
<settings>
<httpWebRequest useUnsafeHeaderParsing="true" />
</settings>
</system.net>

I still don't like it, but thats the proof I needed for the manufacturer.  Thanks again!

Answer 13

I meet the same problem.

The Hello world could not be visited by vs2005 studio.

 

Answer 14

I meet the same problem :(

I m building a project about mobile device application and I wanted to add web reference to tho my project. This problem occured.

How can I solve this problem. There is no web config file on this project ?

Thanks.

Answer 15

The same setting can be applied to your app.config.

Answer 16

Kemal -

Be sure to read these threads completely, your answer is above your question.  But to clairify:

A Windows Application project can be told to parse unsafe headers using an Application Configuration file.  Here is an example routine:

1. In the Solution Explorer, right-click the Project Name
2. Select Add, and then click New Item
3. Select Application Configuration File, you can use the default name App.config, and click Open.  The following code is generated for you in the config file:
• <?xml version="1.0" encoding="utf-8" ?>
  <configuration>
  </configuration>
4. Now modify the pre-generated code to the following:
• <?xml version="1.0" encoding="utf-8" ?>
  <configuration>
     <system.net>
        <settings>
           <httpWebRequest useUnsafeHeaderParsing="true" />
        </settings>
     </system.net>
  </configuration>
5. Save the modified application configuration file

Your Windows Application will now properly parse unsafe headers.

Answer 17

I saw this before but my problem is different from these. My project is mobile device application not windows application. I am working on VStudio.Net 2005. I dont see it with VS.net 2003.You can see screenshot from this link : http://rapidshare.de/files/18495742/error.JPG.html

 

Thanks.

Answer 18

This only works while running in the IDE.

 

The problem still exists when you run the executable created during the build outside of the IDE (release or debug).

 

This problem is a SERIOUS flaw in the design of the the .NET language.  Microsoft can not assume that every page on the net has a correct header.  It is the programmers responsibility to determine what to do.  This flaw has seriously pissed me off.  I have spent days coding an application for a client assuming that I had solved the problem with the addition to the config file only to find that I now cannot deliver the application.

 

I need an IMMEDIATE work around or I am going to have to throw everything I have done away and write the application in another language/platform.

Does anyone have a solution that will work after you build and run outside of the IDE?

 

 

Answer 19

Greg,

The System.Net classes are following the RFC standard.  They support the config setting above to relax the header parsing rules for non-standard servers.

Let's say your app is named xyz.exe, you have an xyz.exe.config with the above configuration in it and the headers are still not being parsed?  What exception are you getting?

Josh

Answer 20

Greg,

I just want to reinforce what Josh has stated.  If the config files exists with the compiled assembly than the problem is resolved in the released executable.

I have successfully created an application that uses the config file posted above.  There must be a problem with your build...  As Josh stated, posting a detailed exception may help us help you.

Also (again reinforcing what Josh stated) this behavior is not a flaw, bug, or error.  It is simply good programming that follows the rules for security reasons.  Internet RFCs exist for a reason, and they should be followed by any engineer that develops Internet enabled applications.

Answer 21

Kemal -

Using a mobile device application doesn't really change anything...

I realize that when you select Add New Item the "Application Configuration File" template is not displayed.  But don't worry about that.  Choose "Code File", name the new item "app.config", and then past the basic config data into the file:

<?xml version="1.0" encoding="utf-8" ?>
  <configuration>
    <system.net>
      <settings>
        <httpWebRequest useUnsafeHeaderParsing="true" />
      </settings>
    </system.net>
  </configuration>

Your Windows Mobile 2005 application will now parse the unsafe headers.

Answer 22

 

I have found why my executable was not working even with the config settings.  It was because I was executing out of the obj and not bin directories.

However, in response to rkimble and the entire Microsoft team that made the decision to make the default setting to block sites with bad headers:

GetResponse call brings back static text, no activex or javascript execution occurs.  Therefore, there is not threat of malicious code executing from this call.  So were is the danger?

So, in my opinion, this is a flaw.  This is Microsoft dictating which sites can be viewed or retreived and forcing all site owners to follow their standards.  I am currently retrieving many pages that have what Microsoft would consider bad headers with no problems.  The pages display in Internet Explorer.  Why is it that they are allowed to bring back these pages, but insist that we can't?  I would think that with my almost 20 years of programming experience that I should be capable of handling these situations the way that I need them to be handled in order to deliver a quality product to my customer.

I sorry if this sounds hostile, it is not meant to be.  I am just stating my opinion supported by some facts.

G McCallum

 

Answer 23

Not to fuel a big debate, as this is not really the place for it, but to be sure other readers understand what is actually happening:

The parsing of message headers can lead to a buffer overflow if those headers are specifically malformed.  Buffer overflows are one of the top exploits that malicious Internet application developers target.  So parsing unsafe headers could indeed lead to problems in applications.

Also understand that the Request For Comments (RFC) documents that define all Internet messages were not written by Microsoft.  The documents are maintianed mostly by IANA, and many that define core IP specifications were written by the Internet Engineering Task Force (IETF) and the Internet Engineering Steering Group (IESG).

So Microsoft did not make they rules - they have only choosen to abide by them, as any good software developer should.

Answer 24

I changed the app conf file but this error has happened again. This is the secreen picture of the error. I couldnt solve it. Please help ?http://rapidshare.de/files/19775848/error.JPG.html

 

 

Answer 25

Your project is a Web Service, not a Windows Application.  Try putting the configuration section in your Web.Config file instead of trying to use an App.Config file.

Answer 26

Mr. rkimble

I tried that you have told me.but  The same error happened again. This the picture of new  one.I want to add web reference with wizard to my MobileDeviceApplication. Is this clear enough ? Thanks for response.

 http://rapidshare.de/files/19961245/error2.JPG.html

 

Answer 27

I couldnt solve the problem. Anyone help me please?

Answer 28

Did you ever figure this out?  Do you realize that it is the Web Service project that needs the configuration entries, not your mobile application?  The Web Service is generating the error so it must be in the Web Service project that you need to add the config info.

 

Answer 29

Hi

Sorry for my English ! I'm French !

I've devel a plug-in for office. And this plug-in is reading a SharePoint WebService !!

I've try your solution, but I don't know in which file I must paste the code you give us !

I have try every web.config file on my server, but none resolved my problem ! And I try to put it in the app.config of the plug-in, but I'm not sure it take it !!

Any idea ?

 

Many thx

Answer 30

As stated in this thread, the configuration entry needs to be in the application's config file, not web.config.  If your plugin is for winword.exe, try making the entry in winword.exe.config.

Josh

Answer 31

I couldnt solve the problem. I put the code into app.config file and then I build the mobile application project succesfuly!. I tried to add web reference and I met the same error: The server committed a protocol violation. Section=ResponseHeader Detail=CR must be followed by LF.  What is this ?  Is this a visual studio 2005 Bug ?  when I try in vs.net 2003 enviroment, this problem is not occured. I want to read an article about this issue. Anyone navigate me ? 

Thanks.

Answer 32

Hi
I don't understand where I can find the file winword.exe.config !! In my VS .Net 2005 Plugin I have only a app.config and this is the only .config file !!
Is it this file ????????

Many thx

Cédric Petter

-- EDIT --

I've found !!
I must write the code in the machine.config of the machine and use the app.config file !!!
and the app.config must be an embedded resource ! and copied to the output dir !!!!
Good bye

Answer 33

My Pc is behind the linux server. I dont have any problems when using direct internet connection. I tried it but I couldnt solve it yet. In this project there is a web service added before.When I click the Right mouse and Update Web Reference link, this error occured.  http://static.flickr.com/71/221926562_8125f2e063_o.jpg

 

 

 

Answer 34

It sounds like you are describing a proxy scenario...  I don't know what else "my pc is behind the linux server" could mean.

I added the web reference you mention to a project and it worked just fine.  There was no error.

I would say that you are behind an internet proxy server and the proxy server is reformatting the headers, causing the CRLF error.  Either remove the proxy server, or replace it with a standards compliant one.

Answer 35

Hi,

My company uses a linux server as a gateway. There is a distribution "Censornet" downloaded from www.censornet.com. There is a proxy server on it but I do not know the details. i.e. I am behind a proxy sever that I can not change and configure. I am using a windows xp sp2 pc. I am runnig VS2005 on it. 

when I want to add web reference to my project, I have the error message like kemal has.

If the CR LF problem is because of the proxy server, I can tell it to the server admin. I read all of the posts under this topic, I added the code you describe but my problem is still going on.

what can I do now?

Answer 36

hi rkimble. I think you cant help us anymore.Am I wrong ?

Answer 37

That appears to be the case.

This thread contains the code that changes the behavior of the CLR to allow unsafe headers.

This thread also explains every possible code file that this code can go in - and when it is appropriate to use each file.

We've also covered which application, in a multi-tier application scenario, is responsible for serving the invalid headers.

Finally, it's been discovered that there may be intermediate servers translating valid headers into invalid ones.

I'm not sure that anyone can help you further; it may be time to lock this thread and let it be...