WCF and proxy firewall troubles

Question

Hi All,

Im having a huge problem trying to get my WCF client to connect to the service when its behind a corporate proxy firewall.

I am using WSHttpBinding with username message security and a custom validator.  The service is hosted in IIS7.0.

The proxy firewall that the client app is behind uses Basic authentication.  The proxy server settings are in IE and when a user attempts to browse to a web site a dialog box pops up prompting the user to enter a username and password.

In the app.config I set useDefaultWebProxy="true" and added the following code:

            System.Net.WebProxy wp = new WebProxy(proxy, true);
            wp.Credentials = new NetworkCredential(username, password);          
            System.Net.WebRequest.DefaultWebProxy = wp;

When I try and call the service I get the following error:


" An error occurred while receiving the HTTP response to http://address_of_service/Service1.svc . This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details."

Digging deeper in the inner exceptions I get the following:

"Unable to read data from the transport connection: an establised connection was closed by the software in your host machine".

I enabled tracing on the service and no entries have been made so the service is not being reached.

To see if I could get past the firewall with a standard web request I tried the following:

            WebRequest wr = WebRequest.Create(urlAddressOfService);    
            wr.Credentials = new NetworkCredential(username,password);              
            System.Net.WebResponse wresp = wr.GetResponse();

 And this worked.


The client binding configuration is as follows:

<binding name="WSHttpBinding_DstServiceContract" closeTimeout="00:01:00"
                    openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00"
                    bypassProxyOnLocal="false" transactionFlow="false" hostNameComparisonMode="StrongWildcard"
                    maxBufferPoolSize="524288" maxReceivedMessageSize="650536"
                    messageEncoding="Text" textEncoding="utf-8" useDefaultWebProxy="true"
                    allowCookies="false">
                    <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
                        maxBytesPerRead="4096" maxNameTableCharCount="16384" />
                    <reliableSession ordered="true" inactivityTimeout="00:10:00"
                        enabled="false" />
                    <security mode="Message">
                        <transport clientCredentialType="Windows" proxyCredentialType="None"
                            realm="" />
                        <message clientCredentialType="UserName" negotiateServiceCredential="true"
                            algorithmSuite="Default" establishSecurityContext="true" />
                    </security>
                </binding>

Im tearing my hair out.  I'd like to add that all works perfectly when a proxy firewall isn't in the way!

If anyone could help out Id be massively grateful.

Answer 1

Hi,

I think you can configure your wcf service endpoint address port to disable the firewall appling on it,this is base on you have got the control of your service security.

Otherwise you need to find out which the ports for publics are and get use of them, normally they are port 80 and 443 via http and https service .
see this blog:http://blogs.msdn.com/drnick/archive/2006/05/01/configuring-wcf-for-nats-and-firewalls.aspx

Thanks
Binze

---

Please remember to mark the replies as answers if they help and unmark them if they provide no help.
Welcome to the All-In-One Code Framework! If you have any feedback, please tell us.

Answer 2

Thanks for the reply.

The problem is not at the service end, the problem is client end.  When I try to run the client application behind a corporate proxy firewall then I can't reach the service.  As in my original post, I can succesfully configure a WebRequest to make it through the firewall OK but the same can not be said of the WCF client.

I had a look at the http packets sent by the WCF client and saw that when an http request is made to connect to the the service the http response header from the proxy firewall contained a '401 unauthorised' error.  When using the WebRequest, the same 401 error was returned but WebRequest responded by sending back the 'proxy-authorization' details in the http header - the proxy firewall then lets the request through.

So why is it a WebRequest responds to a 401 error by supplying the proxy firewall credentials yet the WCF proxy does not?

Answer 3

Hi raptorSW,

After reviewed the first message you posted, I found that there are something here worth noticing:

** The WCF service code you provided used "WebRequest.DefaultWebProxy" to specify the proxy and authentication info, however, it failed with some 401 error.

** The WebRequest code passed the firewall and worked correctly, however, it didn't specify any proxy info. Doesn't that means even if you doesn't specify proxy info(or proxy authentication), it still can reach your service server?

** If the above ones are true, then I think the 401 error is not likely due to the firewall proxy, but the service server's http transport layer authentication. For your WCF service, are you deploying it in IIS server? If so, have you checked whether the service's virtual directory has enabled windows authentication and disabled "allow anonymous"? This will make the service to demand windows authentication at transport layer and your WCF client will surely fail since it use message security and cannot provide any  transport layer authentication info. (Also, your WCF service tracing is focus on Message layer after the messag reach the WCF runtime engine and if it fails at transport layer before WCF runtime can process it, the tracing activity cannot record anything about it).

You can try verifying the things above to see whether it matches your service environment.

---

Please remember to mark the replies as answers if they help and unmark them if they provide no help.

Answer 4

Hi Steven,

Thank you for your response.

For the WebRequest I set the WebRequest.Credentials property to the authorised username and password for the proxy firewall.  This causes an 'authorization: basic' entry to be added to the http request header (I verified this by inspecting the http packets).  I do not need to specify proxy info (other than the authorization credentials) - this is, I assume, picked up from IE.  However, if I do not specify the authorization credentials the webrequest fails.

When I specify proxy information in the WCF code, the http request header does NOT include a 'authorization: basic' header and this appears to be why it's not getting past the proxy firewall.  So I do not understand why the WebRequest supplied the authorization header yet the WCF proxy does not??  I did notice that for the WebRequest a GET is simply used in the http request header (as expected) but when the WCF client proxy first connects a POST is used (presumably to send service credentials).

I have checked in the authorization in IIS and 'allow anonymous' is disabled but Windows Authentication is not in the list.  Besides, I have tried running the client on a different network that is not behind a proxy firewall and it works great.

I assume that the whole proxy firewall causes other people some issues?  How does one deal with the situation where a developed client application may or may not be behind a proxy firewall?  I worry that if I ever do get around this problem, what happens when we deploy the client application and it is behind a different proxy firewall - am I going to go through this headache again?

Thanks for your help.

Answer 5

Hi Raptor,

when setting up your NetWorkCredentials can you please ensure you specify the Domain. So do 

wp.Credentials = new NetworkCredential(username, password, domain);    and NOT

wr.Credentials = new NetworkCredential(username,password);     

If you were passing the domain in the username like "ABC\XYZ" then you will need to split out the domain.

I had similar issue and specifying the domain did the trick.

Cheers,

Dom

Answer 6

As a follow up to the last post I made:

I have noticed that when I change the web request and set WebRequest.Proxy.Credentials to the authorised username and password for the proxy firewall then a 401 error is returned because the "authorization: basic" entry is not present in the http request header.  Yet, as previously stated, when I use WebRequest.Credentials the header appears and it gets past the firewall!

Answer 7

Hi AusDom,

The proxy firewall is not part of a domain and that is why I haven't specified it.

Answer 8

Hi RaptorSW,

As you said that:

=========
I have noticed that when I change the web request and set WebRequest.Proxy.Credentials to the authorised username and password for the proxy firewall then a 401 error is returned because the "authorization: basic" entry is not present in the http request header.  Yet, as previously stated, when I use WebRequest.Credentials the header appears and it gets past the firewall!
============

I think the 401 is not caused by the proxy/firewall, but your webserver which host the WCF service. Because

WebRequest.Credentials is used for authentication with your  target service server instead of the intermediate proxy/firewall.

In addition, I saw you mentioned that for the WCF server, "allow anonymous is disabled and windows auth is not selected", then is basic authentication selected? For test, you can turn on "allow anonymous" for the IIS service virtual directory, I think that will make your client webrequest call pass through even if you do not specify any  authentication info(the 401 is caused by the IIS's authentication rather than the firewall/proxy).

---

Please remember to mark the replies as answers if they help and unmark them if they provide no help.

Answer 9

Thanks Steven,

I have enabled anonymous authentication and it has not made a difference.

I'm pretty convinced it is the proxy firewall causing the 401.  When I change the WebRequest.Credentials to some invalid credentials I get the 401 - changing them back to valid credentials then works.  Also, looking at the http packets that fail to make it through - the http response packets always contain WWW-authenticate: basic with a stated realm of the proxy firewall.  On top of this I am able to access the service when not behind a proxy firewall.

Also worth noting - When I do get the 401 errors they are not restricted to my service - I also get them when trying to access other (well known) sites.

I guess the question is now:  Why does WebRequest.Credentials seem to authorize the proxy firewall yet WebRequest.Proxy.Credentials does not (due to not generating a proxy-authorization header in response to a 401 error)? 

Cheers

Answer 10

OK - a little more progress on this. 

There appears to be an issue with the proxy firewall response depending upon whether the request was made via IE or System.Net.WebRequest.

When I make a request via IE a dialog appears asking for username and password.  Once valid credentials have been entered you are forwarded on to the required destination.  Using HTTP sniffer, I recorded the http packets for a scenario where IE was used:

GET http:// www.msn.com/ HTTP/1.1
Accept: */*
Accept-Language: en-gb
UA-CPU: x86
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; InfoPath.2; .NET CLR 3.0.04506.30; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)
Host: www.msn,com
Proxy-Connection: Keep-Alive

Response

HTTP/1.1 407 Proxy Authentication required
Proxy-Authenticate: Basic realm="Firewall"
Connection: close
Content-type: text/html

The broswer then, as expected, sends out an identical http request but this time with a 'Proxy-Authorization: Basic'

However, when I perform a System.Net.WebRequest in a C# .NET application I get a different response from the proxy.  Specifically, the proxy returns a '401 Unauthorized' along with a 'WWW-authorization: basic' header.  The C# code is below:

            WebRequest wr = WebRequest.Create("http://www.msn.com ");
            System.Net.WebResponse wresp = null;
            System.Net.WebRequest.DefaultWebProxy.Credentials = new NetworkCredential(username, password);
            try
            {
                wresp = wr.GetResponse();
                Stream stream = wresp.GetResponseStream();
                StreamReader reader = new StreamReader(stream, Encoding.UTF8);
                string content = reader.ReadToEnd();
               }
            catch (System.Net.WebException ex)
            {
                Console.Write(ex.Message);
            }

The traced http headers for the above scenario are shown below:

GET http://www.msn.com/ HTTP/1.1
Host: www.msn.com
Proxy-Connection: Keep-Alive

Response:

HTTP/1.1 401 Authentication required
WWW-Authenticate: Basic realm="Firewall"
Connection: close
Content-type: text/html

Because a 407 error is not returned the proxy credentials are not sent to the proxy server and the web request fails.  Setting WebRequest.Credentials to the correct proxy credentials does get past the firewall but I specifically need it to return a 407 error so that I can deal with this in my WCF client.

Does anyone have any idea on why different response headers are generated?

In a WCF client proxy is there a way of setting credentials that respond to a 401 error?

Many thanks.

Answer 11

OK - I think I have got to the root of the problem now.

It would appear that the 401 error is being returned because a 'user-agent' is not being specified in the http request.

When I try the following:


            HttpWebRequest wr = (HttpWebRequest)WebRequest.Create("http://www.msn.com");
            System.Net.WebRequest.DefaultWebProxy.Credentials = new System.Net.NetworkCredential(proxyUsername,ProxyPassword);
            wr.UserAgent = "Mozilla/4.0";            
            System.Net.WebResponse wresp = null;
           
             try
            {
                wresp = wr.GetResponse();
                Stream stream = wresp.GetResponseStream();
                StreamReader reader = new StreamReader(stream, Encoding.UTF8);
                string content = reader.ReadToEnd();

                String st = String.Empty;
            }
            catch (System.Net.WebException ex)
            {
                Console.Write(ex.Message);
            }

It works fine.  A 407 error is returned and the credentials set on the DefaultWebProxy are returned via a 'proxy-authorization' entry in the http request header.

When I do not specify a user-agent then I get the original problem - a 401 error and the DefaultWebProxy credentials are not returned.

Now - the question is how do I get around this in WCF?

1) Is it possible to set a 'user-agent' in the outgoing http headers in WCF?
2) If 1) is possible would it not be better to set a 'proxy-authorize' header in all outgoing http requests?
3) Is there another means around this without setting http headers in WCF?

I could do with (as much as possible) a global solution so that I can get my client to work behind any proxy firewall.  I think for the majority of cases either:

i)  Setting useDefaultWebProxy="true" and useDefaultCredentials="true or
ii)  Setting useDefaultWebProxy="true" and System.Net.WebRequest.DefaultWebProxy.Credentials = new NetworkCredential(username, password);

Would suffice in getting past most proxy firewalls.  I have obviously encountered a problem where the above do not.

Please, please - any help/suggestions would be VERY welcome.  I have been trying to get a solution to this problem for a long time now.

Thanks

Answer 12

Hi raptorSW,

Thanks for your followup and glad that you've made much progress.

The behavior you found is really unexpected as the "useragent" http header make the proxy/firewall return such kind of misleading error.

So currently, what you need is programmatically set the "user-agent" header for your WCF request. I think the following article will help:'

#Manipulating HTTP Headers in your WCF Message
http://kennyw.com/indigo/153

---

Please remember to mark the replies as answers if they help and unmark them if they provide no help.

Answer 13

Thanks Steven,

I have tried to manipulate the http header by adding a 'user-agent' but it did not appear in the http request header.  Here is my code:

            DstProxy = new DSTdemo_WPF.DST_ServiceProxy.DstServiceContractClient("WSHttpBinding_DstServiceContract");           
            DstProxy.ClientCredentials.UserName.UserName = username;
            DstProxy.ClientCredentials.UserName.Password = password
            
            HttpRequestMessageProperty httpRequestProperty = new HttpRequestMessageProperty();
            httpRequestProperty.Headers[HttpRequestHeader.UserAgent] = "Mozilla/4.0";

            using (OperationContextScope scope = new OperationContextScope(DstProxy.InnerChannel))
            {              
                OperationContext.Current.OutgoingMessageProperties[HttpRequestMessageProperty.Name] = httpRequestProperty;
               
                try
                {
                    DstProxy.InvokeService();
                }
                catch (Exception ex)
                {
                    Console.Write(ex.Message);
                }
            }

the http request packet contains:

POST http://myService/Service1.svc HTTP/1.1

Content-Type: application/soap+xml; charset=utf-8

Host: myService

Content-Length: 1130

Expect: 100-continue

Proxy-Connection: Keep-Alive

As you can see - no user-agent header has been added? 

Any ideas why?

Answer 14

This post continues here: http://social.msdn.microsoft.com/Forums/en-US/wcf/thread/a47c50b6-2fbb-4249-b33f-8d656839efce

I have finally reached a conclusion.  It would appear there is no way around this firewall in my case.

Hope this helps out someone who has a similar problem.

Answer 15

we had and still have/see very similar issues when talking to a firewall but from java clients. In general; our FW was starting to negociate for the authentication shema to be used and if no hint was given up on initial call by the client such as http.auth.preference="BASIC" the FW expected first a NTLM authentication would suffice, which a Java client by nature (at that time) was unable to cooperate with. Hence it failed. Seein that all, we finally learned that our FW wants NTLM but would accept BASIC as well if we request for that with the first initial request. But how to pass this information is a bit of magic in most and all environments we have seen up to date.

In such a situation with a FW one needs to consider that any packages sent toward an external service may need to send 

a) http.proxy.User="proxyuser"
b) http.proxy.Password="proxyUsersPassword"
c) http.proxy.Server="xyz.proxyServer.com"
d) http.proxy.Port=8080
e) http.auth.Preference="BASIC"
to pass the FW with outgoing requests.

And then you may ask:
Does my web service demand similar credetials or what are all the credentials used at the web service and when and how do we pass it.

Do we need to initially negoiate with the FW, a) in a separate call, or b) in a consecutive call, ... in other words ... what is the proper protocoll sequence to be used toward any FW, and hence, to be sniffed by sorts of network analyzers, what can be expected to be seen.

ANd if the web service needs credentials too, how are they added to your outgoing request by i.e. :NET frame work routines?

We found, that very often client SW is able to deliver a) to d) above but is in general unable to deliver the wanted e) which is a hint to the FW/Proxy-Server about authorisation protocols to be used.

In my mind, these 5 elements belong together at all time for a) a proxy server and b) any service delivering server what ever it is IIS or AXIS2 or Metro of Glassfish.

Josef

---

Sepp