How do I enable useUnsafeHeaderParsing from code? (.NET 2.0)

Question

Hi,

We have several webservers that do not follow the correct RFC specification when performing some specific HTTP web requests, in order to communicate with them we need to enable unsafe header parsing. Changeing the webservers to respond correctly is NOT an option in our case.

According to documentation, adding the following lines to the app.config enables the unsafe header parsing.

<system.net>
<settings>
<httpWebRequest useUnsafeHeaderParsing = "true"/>
</settings>
</system.net>

This works nicely for regular applications, however, the problem is that our .NET code is exposed and run as a COM object. And adding a .config file to .NET assemblies doesn't work, neither when being loaded by COM or if the .NET assembly is loaded and used from another .NET application.

Since the application that loads our .NET COM object isn't a .NET application it can not have it's own app.config file.

So how do I enable unsafe header parsing in code?

In .NET 1.1 the unsafe header parsing flag could be changed using reflection, however this flag does not appear to be in the same place in .NET 2.0.

Cheers,
RedZ

Answer 1

Ok, I figured it out. It can be done in a similar way as with .NET 1.1. However it would be a lot nicer and safer if there had been a public API for doing this in code, instead of using reflection.

A lot of people have to deal with a large number of legacy systems that can't be corrected to use the proper RFC HTTP responses and for them this is a real issue. Idealy the use of safe or unsafe parsing should be configurable in the HTTP request and not on a global level.

Anyway, here's the code for hacking/fixing it in .NET 2.0

(Add System.Configuration as a reference to your project.)

public static bool SetAllowUnsafeHeaderParsing20()
{
  //Get the assembly that contains the internal class
  Assembly aNetAssembly = Assembly.GetAssembly(typeof(System.Net.Configuration.SettingsSection));
  if (aNetAssembly != null)
  {
    //Use the assembly in order to get the internal type for the internal class
    Type aSettingsType = aNetAssembly.GetType("System.Net.Configuration.SettingsSectionInternal");
    if (aSettingsType != null)
    {
      //Use the internal static property to get an instance of the internal settings class.
      //If the static instance isn't created allready the property will create it for us.
      object anInstance = aSettingsType.InvokeMember("Section",
        BindingFlags.Static | BindingFlags.GetProperty | BindingFlags.NonPublic, null, null, new object[] { });

      if (anInstance != null)
      {
        //Locate the private bool field that tells the framework is unsafe header parsing should be allowed or not
        FieldInfo aUseUnsafeHeaderParsing = aSettingsType.GetField("useUnsafeHeaderParsing", BindingFlags.NonPublic | BindingFlags.Instance);
        if (aUseUnsafeHeaderParsing != null)
        {
          aUseUnsafeHeaderParsing.SetValue(anInstance, true);
          return true;
        }
      }
    }
  }
  return false;
}

Cheers,
RedZ

Answer 2

 

Thankyou so much for including the code... Saved me some time... It took me a while to get this far and recognise that this was my problem... It was only when I started sniffing the packets with Ethereal that I realised I was getting an OK response.  It was just that the embedded web server of the device inserts an additional header which .Net 2.0 didn't like... Everyone then assumes that you're using ASP which I'm not either...

Thanks once again...

Answer 3

I very much could use a code example for .net 1.1 using reflection of this same task!

Thanks

Answer 4

Thank you thank you thank you! This worked great ... now if only this article could be linked directly from the "useUnsafeHeaderParsing" element page in MSDN, it would have saved me so much time.

Now for some other keywords to help others get this via Google...
- This solution shows how to set HttpWebRequest.useUnsafeHeaderParsing at runtime.
- Set useUnsafeHeaderParsing without app.config
- GetResponse() returns null
- HttpWebRequest.GetResponse exception message: "The server committed a protocol violation. Section=ResponseHeader Detail=CR must be followed by LF"


cheers,
SirThomas

Answer 5

Here is the code for VB.Net:

Imports System.Reflection

Private Sub SetAllowUnsafeHeaderParsing20()
Dim a As New System.Net.Configuration.SettingsSection
Dim aNetAssembly As System.Reflection.Assembly = Assembly.GetAssembly(a.GetType)
Dim aSettingsType As Type = aNetAssembly.GetType("System.Net.Configuration.SettingsSectionInternal")
Dim args As Object()
Dim anInstance As Object = aSettingsType.InvokeMember("Section", BindingFlags.Static Or BindingFlags.GetProperty Or BindingFlags.NonPublic, Nothing, Nothing, args)
Dim aUseUnsafeHeaderParsing As FieldInfo = aSettingsType.GetField("useUnsafeHeaderParsing", BindingFlags.NonPublic Or BindingFlags.Instance)
aUseUnsafeHeaderParsing.SetValue(anInstance, True)
End Sub


I had to also add a reference in the project to System.Configuration.

All credit is due to UncleRedZ - excellent!  Saved me a lot of work.  I agree with him that this should be much easier to do directly with the WebRequest classes instead of going through all of this.  Unix based systems seem to do this.

Answer 6

Thanks to uncleRedZ and CharlieStrause.

This problem drove me crazy for 2 days.

This should be part of the webrequest class!!!!!!!!!!

 

Answer 7

Thank you UncleRedZ, it is very helpful and saved me time from composing my own web requests using Sockets in some case.

Answer 8

Does anyone have any idea why this wouldn't work?
I'm accessing a web-server which has an expired SSL certificate.
Despite setting useUnsafeHeaderParsing to true (which according to documentation should ignore validation errors) I still get a WebException exception with "TrustFailure".
Is this not a "validation error"?
Should I be barking up another tree?

Thanks,

Answer 9

Hello

 

I don't think this has to do with unsafe HTTP headers.
Well, I think you should find another solution.

 

--

Elias

 

Answer 10

The word 'unsafe' makes me uneasy here.
Is the actual implementation safe?
Am I more likely to get buffer overflowed if I use the unsafe version? Has that code been tested less?

Answer 11

Last time I looked up that parameter, I found an explanation from Microsoft of what that means eactly. So Microsoft's own explanation might help you to decide what "unsafe" means. As I recall, it had something to do with a technicality in implementing HTTP protocol. So is it really fairly safe, and just a technical detail of protocol handling? Or could it really lead to an exploit? You'll have to find a better explanation somewhere. If so, make a link to it from here.

Answer 12

Thanks for this post. It helped our team in solving the violation issue reported for quite some time.

Answer 13

Where should I implement this method and where should it be called?

Answer 14

Hi Prathiba, do you know where in the code and when this method should be called?

Answer 15

Excellent!!!

Many thanks for this solution!

Answer 16

I only signed up to say THANK YOU SO MUCH. I don't necessarily understand why it works but it fixed my issue!

Answer 17

Thank you very much UncleRedZ.  This worked perfectly.  And yes, this should have been incorporated as part of the HttpWebRequest class.

 

For those who are asking where and when in the code the above function should be called: just call it at anytime before you do request.GetResponse())

 

Also, note that you need:

using System.Reflection;

Answer 18

OKay so i tried the code as is, and made a call to the function right before request.GetResponse() - however, i continue to get the error.

 

My app setup is...A Webservice calls a worker dll and from withing the worked dll, at runtime we invoke another dll which actually makes the WebRequest.

 

SHould this function be called before i load the final dll? RIght now i am calling the function from within the final dll.

Answer 19

Thanks for the code sample, I used to to solve my problem and to create a variation of it that is much simpler and shorter.

You can read more details about it here: http://o2platform.wordpress.com/2010/10/20/dealing-with-the-server-committed-a-protocol-violation-sectionresponsestatusline

Answer 20

Thanks for this post.  But I am looking for a C++ solution.

How would I get at the setting so I could  use it in
    IWinHttpRequest *pIWinHttpRequest;
    ...
    pIWinHttpRequest->put_Option(...)
?

Any help appreciated.

Answer 21

Thank you! It worked for me even though it looks a bit complicate;)