locked
401 Unauthorized on new website always solved after reboot IIS7.5 RRS feed

  • Question

  • User1802555839 posted

    Hi all,

    Phenomenon:
    When I create a new website in IIS, and I want to use the website using windows authentication, I'll always get a "401 unauthorized". The only way to solve this is a reboot of the server. Everytime I want the Integrated windows authentication to work for a new site, I've got to do a reboot. Once this single reboot is done, it's solved (like forever).

    Config:
    Applicationpool:
    .NET: v4.0, Managed Pipeline Mode: Integrated, Identity "applicationpoolidentity" (i did not modify the advanced settings = still installation default)
    Website:
    Authentication: Windows Authentication Enabled with "enable Kernel mode authentication", extended protection Off, Providers: Negotiate, NTLM
    All the rest is default.
    Server:
    Windows 2008 R2 64bit, Applications are developped in VS 2010.
    Domain member, enabled the machine account for delegation (all protocols) in the AD, set an SPN for one dummy application "HTTP/hello-world".
    DNS configs:
    The url's I use is http://hello-world. The AD domain is "admin.mycompany.internal", so the server is knwon as "iisserver1.admin.mycompany.internal" and the applications dns is know as "hello-world.mycompany.int".

    Any clues what is missing here?

    Regards,
    David.

    Monday, January 31, 2011 10:03 AM

Answers

  • User690216013 posted

    Did you spend some time on IIS logs and Security event log? When you see a 401 you should at least see whether it is 401.1 or 401.3. Then Security event log may tell you what happened at that time.

    Regards,

    • Marked as answer by Anonymous Tuesday, September 28, 2021 12:00 AM
    Tuesday, February 1, 2011 7:25 AM

All replies

  • User690216013 posted

    Did you spend some time on IIS logs and Security event log? When you see a 401 you should at least see whether it is 401.1 or 401.3. Then Security event log may tell you what happened at that time.

    Regards,

    • Marked as answer by Anonymous Tuesday, September 28, 2021 12:00 AM
    Tuesday, February 1, 2011 7:25 AM
  • User1802555839 posted

    Hi,

    Thanks for answering. I was busy on another project and did not have time yet to go in detail. Sorry for the late answer.

    Today, I had to reconfigure another website it's applicationpool (running asp.net 4.0 Integrated) to classic mode (4.0), enable also the  "asp impersonation" in the authentication properties, to allow impersonation to be used against a webservice on another server.
    Again, after doing this, I was unable to access the website, same phenomenon as when creating a new site.

    I checked the IIS log of that website:

    2011-02-23 16:36:52 10.187.137.28 GET / - 80 - 10.197.36.38 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+Trident/4.0;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729;+.NET4.0C;+.NET4.0E) 401 2 5 5
    2011-02-23 16:36:52 10.187.137.28 GET / - 80 - 10.197.36.38 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.1;+Trident/4.0;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729;+.NET4.0C;+.NET4.0E) 401 1 2148074254 18

    I stopped IIS Admin service, The WWW service, started both again => no effect
    I stopped/started/recycled the application pool related to this site => no effect

    I finally restarted my server, and the issue was gone.
    Very strange :S !

    I have a solution (reboot), but still I'm wondering what I'm doing wrong, or if there's some magic command I've got to do to get this thing working without a reboot? Somekind of credential caching mechanism going wrong on the server?

    Regards,

    Wednesday, February 23, 2011 11:58 AM
  • User1802555839 posted

    I've changed the way how I'm creating new websites.

    During the creation, the check "start web site immediately" is enabled (by default).
    I'm now deselecting this, setting my applicationpool correct first (to 4.0), my authentication protocols, etc...
    After that, I'm starting the website, and it works.

    mmm
    anyone had the same expierence?

    Regards,

    Thursday, February 24, 2011 12:00 PM
  • User355265702 posted
    Hi, I am having exactly this issue on three separate web servers, however I was not able to solve it using your suggested steps. Could you please outline in a little more detail the ordered steps you took? Many thanks
    Monday, June 11, 2012 4:13 AM
  • User1802555839 posted

    I think I know what to do to prevent this behaviour (not tested yet though)

    You have to use A-records in DNS instead of CNAME-records (e.g. myapp has IP 10.1.1.1), and need to register the short dns name as well as the fully domain name, eg myapp.mycompany.local (with the setspn command).

    Tuesday, June 12, 2012 7:29 AM
  • User355265702 posted
    I replaced my CNAME record with a A Record and it started working. No more prompts, auto authentication working perfectly. However I did not run any commands with setspn. I'm happy it is working but do you know why this happens?
    Tuesday, June 12, 2012 8:49 AM
  • User-619846739 posted

    Hello Gray,

    I haven't run into this specifically so I don't have a good answer. I can only brainstorm. One thing to check first is that you don't have forms auth and windows auth both enabled. That isn't supported in IIS7+.

    I can't even guess what the difference would be between a cname and alias unless there's a bug that can't follow the alias correctly. I've never seen something like that, but maybe that's what your situation is.

    Do you have 2 or more DNS servers set? I assume so. Can you confirm that all of them are working correctly and that it's not getting bad results some of the time?

    For the SPN I would assume that it's a hard failure if it's wrong and not something that would require a reboot. Are you just dealing with local content, like a hello world on the server?  If so, I don't think that the SPN comes into play.

    What are you using for the app pool identity?

    Tuesday, June 12, 2012 9:40 AM
  • User355265702 posted

    Hi Scott,

    Thanks for the help via twitter and here.

    This issue is intermittent, goes away for a week or two after a reboot and then comes back, but only for new websites, existing sites which had the issues before the reboot do not ever have it again.

    We have this issue on three webservers on three different sub domains, each is completely independent.

    Each web server has 2 DNS servers set which are all working perfectly with no bad results, our infrastructure team have confirmed this.

    I have replicated this with a website containing only a single helloworld.txt file.

    I have run ProcMon and established that with the CNAME and A Record there are no access denied errors in either case.

    All of the below is the same whether using a CNAME or A Record.  It works with A Record, but am prompted for credential and eventually given 403 with CNAME.

    ApplicationPoolIdentity has read access to c:\inetpub\my site

    Here is the section from applicationhost.config (with details redacted):
    <applicationPools>
                   <add name="my site" autoStart="true" managedRuntimeVersion="v4.0">
                    <processModel identityType="ApplicationPoolIdentity" />
                </add>
    </applicationPools>

    <sites>
       <site name="my site" id="17" serverAutoStart="true">
                    <application path="/" applicationPool="my site">
                        <virtualDirectory path="/" physicalPath="c:\inetpub\my site" />
                    </application>
                    <bindings>
                        <binding protocol="http" bindingInformation="*:80:mysite" />
                        <binding protocol="http" bindingInformation="*:80:mysite.uk.example.com" />
                        <binding protocol="http" bindingInformation="*:80:mysite.il.example.com" />
                    </bindings>
                    <traceFailedRequestsLogging enabled="true" />
                </site>
    </sites>

    <location path="my site">
            <system.webServer>
                <security>
                    <authentication>
                        <anonymousAuthentication enabled="false" />
                        <windowsAuthentication enabled="true" />
                    </authentication>
                </security>
            </system.webServer>
    </location>

    and from the web.config:
        <authentication mode="Windows" />

    Extract from failed request (using CNAME) from log file (with details redacted):
    #Fields: date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-win32-status sc-bytes cs-bytes time-taken
    2012-06-11 12:55:34 W3SVC11 MYSERVER 101.63.201.69 GET /hello-world.txt - 80 - 171.21.101.125 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.1;+Trident/4.0;+SLCC2;+.NET+CLR+2.0.50727;+.NET+CLR+3.5.30729;+.NET+CLR+3.0.30729;+Media+Center+PC+6.0;+InfoPath.3;+Tablet+PC+2.0;+.NET4.0C;+.NET4.0E) - - mysite 401 5 1509 383 0
    2012-06-11 12:55:34 W3SVC11 MYSERVER 101.63.201.69 GET /hello-world.txt - 80 - 171.21.101.125 HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.1;+Trident/4.0;+SLCC2;+.NET+CLR+2.0.50727;+.NET+CLR+3.5.30729;+.NET+CLR+3.0.30729;+Media+Center+PC+6.0;+InfoPath.3;+Tablet+PC+2.0;+.NET4.0C;+.NET4.0E) - - mysite 401 2148074254 1509 4354 15

    Tuesday, June 12, 2012 10:41 AM
  • User355265702 posted

    Formatting fixed!

    Tuesday, June 12, 2012 10:47 AM
  • User1802555839 posted

    Hi,

    The setspn command is necessary if you want to work with impersonating the client in your asp.net code, if you want your webapplication (application pool) to access other resources in the domain using the user it's kerberos authentication

    Reading: http://msdn.microsoft.com/en-us/magazine/cc163740.aspx (it's from windows 2003 but it's still the same in IIS 7.x) and also http://support.microsoft.com/kb/929650 (How to use SPNs when you configure Web applications that are hosted on Internet Information Services)

    I'm happy that this worked out fine for you!

    PS: we're also running several domain zones in our internal network, but the CNAME records were only defined in the same zone as the Active Directory zone, in which the clients and IIS servers are member off. I've seen the usage of A-records by an MVP doing a sharepoint implementation, specificly asking for A-records to avoid issues.

    Tuesday, June 12, 2012 11:25 AM
  • User355265702 posted
    Thanks for the reply, that makes sense why the SETSPN command was not necessary. My web application does not access any resources on the domain, only a MSSQL database via a SQL login.
    Tuesday, June 12, 2012 11:28 AM
  • User-619846739 posted

    Something interesting in your logs is the 401.5 and then 401.2.  A 401.5 is authentication failed by ISAPI/CGI application.  Which ISAPI extensions do you have? Can you disable them for your failed site and see if it works?  You may find that this issue tracks back to one of them.  Since even hello-world.txt fails it doesn't sound like it's ASP.NET causing the issue, and in Integrated mode ASP.NET doesn't run as an ISAPI extension.

    The 401.2 is Logon failed due to server configuration (http://support.microsoft.com/kb/943891) so that also suggests that it's a configuration auth situation.

    I can't tell from the configuration whether you have forms auth and windows auth both enabled. You can confirm that from the Authentication icon at the site level.

    The A/CNAME test that you did, are you just talking which domain name you use when accessing the site? for example if http://mysite.uk.example.com is an alias then it fails, but if it's an A record then it works?  If that's the case then it does sound like it must be some ISAPI extension which does a lookup on the domain name before it continues along to handle the request normally.

    Tuesday, June 12, 2012 1:30 PM
  • User355265702 posted

    Which ISAPI extensions do you have? Can you disable them for your failed site and see if it works?
    I'll check this tomorrow morning as soon as I get in.

    I can't tell from the configuration whether you have forms auth and windows auth both enabled. You can confirm that from the Authentication icon at the site level.
    Forms, anonymous and all other forms of authentication are disabled, only windows authentication is enabled.

    The A/CNAME test that you did, are you just talking which domain name you use when accessing the site? for example if http://mysite.uk.example.com is an alias then it fails, but if it's an A record then it works?

    Exactly. 
    CNAME mysitecname -> servername.uk.example.com fails
    A Record mysitearecord -> [server ip address] works
    Tuesday, June 12, 2012 5:15 PM
  • User-619846739 posted

    IIS itself shouldn't even use the domain name except for the site binding, and it doesn't need to do a lookup to the DNS server.  You can use a made up top level domain (like 'testing') if you point a hosts entry to it.  So, it's really sounding like another ISAPI component coming into play that tries to do resolve the domain name.

    Also look into any advanced logging add-ons to see if it's trying to resolve the domain name.

    Tuesday, June 12, 2012 5:23 PM
  • User355265702 posted

    I'm afraid I don't know much about ISAPI components, so I thought the best I could do it show you the settings.  Any suggestions?

    I think it's relevant, the app pool is .NET 4.0 Integrated ApplicationPoolIdentity.

    ISAPI & CGI Restrictions - IIS Server Level

     ISAPI & CGI Restrictions - IIS Server Level.PNG

    ISAPI Filters - Server Level
    ISAPI Filters - IIS Server Level

    ISAPI Filters - IIS Site Level
    ISAPI Filters - Site Level

    Wednesday, June 13, 2012 5:06 AM
  • User1802555839 posted

    IIS itself shouldn't even use the domain name except for the site binding, and it doesn't need to do a lookup to the DNS server.  You can use a made up top level domain (like 'testing') if you point a hosts entry to it.  So, it's really sounding like another ISAPI component coming into play that tries to do resolve the domain name.

    Also look into any advanced logging add-ons to see if it's trying to resolve the domain name.

     

    Well that is true, but we've seen in testing and debugging this "phenomenon", that when we added the mysite.company.local address in our host file locally, the application worked again properly. We thought it was the browser behaving differently, but in fact, we changed our record on our pc to an A-record.

    Interesting issue this A-record versus CNAME-record!!!

    Wednesday, June 13, 2012 9:59 AM
  • User-619846739 posted

    @dabuurkes, that is interesting indeed. I can't imagine what DNS lookup would be performed on the website binding.

    @GrahamKing1983, you don't have anything out of the ordinary. Using Integrated Mode, it's not usually any of those for a .txt file.

    I'll ask around and see what I can find out.

    Wednesday, June 13, 2012 10:39 AM
  • User-2064283741 posted

    I must say I don't buy the CNAME versus A record setup.

    I suspect there is something wrong with your DNS configuration maybe there is no DNS glue or something.

    CNAME mysitecname -> servername.uk.example.com fails
    A Record mysitearecord -> [server ip address] works 

    So what does the servername.uk.example.com resolve too??

    In your example it doesn't resolve to an IP.

    What is your real world site name I have some tools to check it. DNSstuff.com is good but behind a paywall for the decent tools

    Also what DNS software are you running? 

    Wednesday, June 13, 2012 4:32 PM
  • User-2064283741 posted

     Also are there any proxies in the way your client end to the server? Proxies can play havoc with DNS if you don't know what is happening.

    Wednesday, June 13, 2012 4:37 PM
  • User355265702 posted


    CNAME mysitecname -> servername.uk.example.com fails
    A Record mysitearecord -> [server ip address] works 

    So what does the servername.uk.example.com resolve too??

    In your example it doesn't resolve to an IP.


    Here's the nslookups (redacted)


    C:\Users\myuser>nslookup
    Default Server:  ukdc1.uk.example.com
    Address:  173.21.251.101

    > mysitecname
    Server:  ukdc1.uk.example.com
    Address:  173.21.251.101

    Name:    servername.uk.example.com
    Address:  11.67.154.47
    Aliases:  mysitecname.UK.example.com

    > servername.uk.example.com
    Server:  ukdc1.uk.example.com
    Address:  173.21.251.101

    Non-authoritative answer:
    Name:    servername.uk.example.com
    Address:  11.67.154.47

    > mysitearecord
    Server:  ukdc1.uk.example.com
    Address:  173.21.251.101

    Name:    mysitearecord.UK.example.com
    Address:  11.67.154.47

    >What is your real world site name I have some tools to check it. DNSstuff.com is good but behind a paywall for the decent tools


    It's an internal web site so I'm afraid your tools won't help but thanks for the offer.

    Also what DNS software are you running? 


    It's a Windows Server 2003 domain so that's running the DNS.

     Also are there any proxies in the way your client end to the server? Proxies can play havoc with DNS if you don't know what is happening.

    No proxies between client and server.

    Thursday, June 14, 2012 5:24 AM
  • User-619846739 posted

    Gray and Dabuurkes,

    I checked with one of the IIS team and he confirmed that no DNS lookup is performed on the binding.  However, he had a couple good recommendation.

    Can you confirm that you haven't set enableReverseDns?  You will see if it's set by running this command:

    c:\windows\system32\inetsrv\appcmd list config "Default Web Site" -section:system.webServer/security/ipSecurity

    Just swap out the site name. Also, to really track this down the best way will be to get a network track with the alias and with the A record. When you run into this next can you get a network trace with Netmon or Wireshark?  Then just update to an A record and get another trace.

    Thanks,

    Scott

    Thursday, June 14, 2012 9:40 AM
  • User355265702 posted

    Hi Scott,

    Here's the output from the appcmd command.

    C:\Users\myuser>c:\windows\system32\inetsrv\appcmd list config "mysite" -section:
    system.webServer/security/ipSecurity
    <system.webServer>
      <security>
        <ipSecurity>
        </ipSecurity>
      </security>
    </system.webServer>

    No enableReverseDns here.

    I'll get a Wireshark network track done tomorrow.

    Thursday, June 14, 2012 12:07 PM
  • User-619846739 posted

    Thanks for the confirmation. It must be something else. It was a stretch, but worth confirming.

    Thursday, June 14, 2012 12:10 PM
  • User1802555839 posted

    I executed this command on all my websites, they all return this: 

    <system.webServer>
      <security>
        <ipSecurity allowUnlisted="true">
        </ipSecurity>
      </security>
    </system.webServer>

     Conclusion: no, we didn't set enableReverseDns.

    Also, there is NO proxy, neither Web Application firewall, neither local firewall or custom isapi filters between the client and the server

    Thursday, June 14, 2012 12:15 PM
  • User-619846739 posted

    It's not enabled by default so that makes sense, and it does rule that out at least.

    Thursday, June 14, 2012 12:19 PM