locked
Azure Websites 500 (URL Rewrite Module Error) RRS feed

  • Question

  • As of today, our users have been receiving this a blue 500 error on our Azure Website - even though friendly errors are turned off in settings. In the error console I see:
    Failed to load resource: the server responded with a status of 500 (URL Rewrite Module Error.)

    We did use the IIS Rewrite module to force HTTPS on all users, but I removed it and the error remains. The strange thing is, the error only occurs when:
    1. There is a colon (:) in the URL's querystring
    2. The HTTPS protocol is used (HTTP is fine)
    For example, passing a JSON object like this (Chrome automatically encodes quotes, but not { or : chars)
    http://myapp.azurewebsites.net/?test={%22key%22:%22value%22}

    This has to be something which has changed recently, as it worked fine up until today. I can replicate this consistently on HTTPS, but not on HTTP at all.

    Our application is an ASP.NET MVC 5 (.NET 4.5) application, if that makes any difference.

    Monday, March 16, 2015 3:07 PM

Answers

  • @ajason: the issue should be globally fixed now. Please confirm that everything is good with your site.

    Sorry that it took so long!

    Friday, March 20, 2015 8:17 PM

All replies

  • Could you share your site name, either directly or indirectly?

    I tried reproing on an empty site, but that worked fine, e.g. https://testcolon.azurewebsites.net/?test={%22key%22:%22value%22}

    What happens if you try creating an empty site in the same region as your problem site? Trying to determine if the issue relates to the content of the site, or to something outside the site.

    Monday, March 16, 2015 4:08 PM
  • Hi David,

    I tried cloning the site onto another "free" instance and there is no issue - even though it's just a direct FTP copy of the files.

    The cloned site is https://qstringtest.azurewebsites.net/Chrome/Get?test={%22key%22:%20%22value%22} (it's Chrome-only) and is hosted on the same resource group as our "live" website if that helps?

    The "live" site is currently redirecting users from HTTPS to HTTP to circumvent the problem.

    Monday, March 16, 2015 4:28 PM
  • Ryan, I see 7 free sites in there, so it's no clear which one has the issue. Is the site that has the issue also a Free site?
    Monday, March 16, 2015 4:40 PM
  • We are also seeing this on our azure website. We haven't done any recent deployments and lots of end users are seeing this... Also, we are not even using IIS Rewrite ( https://github.com/exceptionless/Exceptionless )


    -Blake Niemyjski (Software Development Engineer)


    Monday, March 16, 2015 4:46 PM
  • Please note that we are running multiple (standard) instances. Everything seems to work but the CORS Preflight requests are failing.

    -Blake Niemyjski (Software Development Engineer)

    Monday, March 16, 2015 4:51 PM
  • Seems like this could have  something to do with the query string value

    https://api.exceptionless.io/api/v2/stacks/frequent?filter=type:error

    causes that blue screen but 

    https://api.exceptionless.io/api/v2/stacks/frequent?filter=typeerror

    works.

    Monday, March 16, 2015 4:58 PM
  • We now understand the issue. It will happen when the following conditions are met:

    1. The site uses IP based SSL
    2. The site is accessed over https
    3. The request has a query string which contains a colon character
    Monday, March 16, 2015 5:01 PM
  • Any eta on a fix?
    Monday, March 16, 2015 5:02 PM
  • We are facing the same problem. We host our API on Azure Website, and a request like https://api.zoom2u.com/breeze/courier/AcceptDelivery?BookingId=25248&PickupETA=2015-03-16T14:09:00.000 is consistently failing with error below. This has starting failing only today, have been running this fine for a few months:

    Error 500 - URL Rewrite Module Error

    <p>The Azure Web Site has experienced an unexpected service error. We apologize for the inconvenience. Please try to reload the page or visit it again soon. </p>
                    <p>If you are the Administrator of this web site, please visit the 
                        <a href="http://www.windowsazure.com/en-us/support/service-dashboard/">Azure Service Dashboard</a> for information about the status of Azure Services.
                    </p>
                    <p>    You can also visit the 
                        <a href="http://social.msdn.microsoft.com/Forums/windowsazure/en-US/home?forum=windowsazurewebsitespreview">Azure Web Sites forum</a> to discuss this.
    Monday, March 16, 2015 5:02 PM
  • No clear ETA yet, but I would estimate that it could take a couple of hours. This is a very rough estimate.

    Apologies for the downtime. We're trying to address the issue as quickly as we can.

    Monday, March 16, 2015 5:15 PM
  • We have been facing the same issue since early afternoon. However, not all our Websites are affected. Luckily some of our sites/services still work.

    Regards,
    Rainer.


    Rainer

    Monday, March 16, 2015 5:27 PM
  • No clear ETA yet, but I would estimate that it could take a couple of hours. This is a very rough estimate.

    Apologies for the downtime. We're trying to address the issue as quickly as we can.

    Thanks a lot David, appreciate the quick response. Certainly had me scratching my head first thing on a Monday morning :)
    Monday, March 16, 2015 5:38 PM
  • While we are working on the fix the quick mitigation to get the sites working again is to temporarily switch from using IP SSL to SNI SSL:

    1. If the site's domain name with IP SSL is a CNAME to <sitename>.azurewebsites.net (as it is the case for api.exceptionless.io) then you can just switch that domain name to SNI SSL by using Azure Portal.

    2. If the site's domain name with IP SSL is a A RECORD to VIP address, then you can still switch that domain name to SNI SSL but then you should also quickly update the A RECORD to point to the default Azure Web Sites VIP that you can get from Azure Portal.

    The downside of this change would be that some clients who use Windows XP with IE6 will start seeing certificate warning when browsing to your site. But that is hopefully a very small percentage of your site's visitors.

    Once we have released a fix you can switch back to using IP SSL.


    http://ruslany.net/

    Monday, March 16, 2015 5:54 PM
  • Thanks that worked!! By the way, I had to use the old portal as the new portals save button did nothing (but discard did).
    Monday, March 16, 2015 6:08 PM
  • We (www.box-planner.com) are having the same problem. Please keep us updated on the status for the fix.

    Monday, March 16, 2015 6:14 PM
  • @boxplanner: did you try the suggested workaround from Ruslan?

    Assuming the workaround is effective for everyone, we are now focusing on the right fix, rather than the quick patch that we were initially looking at.

    Monday, March 16, 2015 6:20 PM
  • Workaround did the job for our site, thank you.

    Regards,
    Rainer.


    Rainer

    Monday, March 16, 2015 6:24 PM
  • Hi, we did not try the workaround so far. If its only a matter of hours I would prefer not changing the DNS settings. If it takes longer then I will anyway have to. What will be the expected deployment of the planned fix ?
    Monday, March 16, 2015 6:24 PM
  • @boxplanner: it will likely take more than a few hours to get the final fix in, so I would suggest using the workaround. We would have used the quick patch approach in the absence of a workaround, but to limit the risk, we are now focusing on the final fix, and that does take a bit longer (could be a day or two).

    David

    Monday, March 16, 2015 6:30 PM
  • Ok, then we will try the workaround. I will let you know.
    Monday, March 16, 2015 6:32 PM
  • Hi, Workaround worked for us as well (at least until now it seems so). Thanks for the fast help,

    box planner

    Monday, March 16, 2015 6:43 PM
  • Unfortunately the workaround will take us a while.  We don't have access to the DNS settings, and it will take some time to get in touch with the right person to do that for us. 

    Could you please just post an updated ETA if/when you have one?  And thanks so much for the quick diagnosis!

    Monday, March 16, 2015 7:21 PM
  • @BethGVinson: it could be a day or two before we are able to deploy the full fix.
    Monday, March 16, 2015 10:53 PM
  • This fix worked for me as well. Thanks.
    Tuesday, March 17, 2015 2:56 AM
  • @Beth: Could you share your site name, either directly or indirectly?

    That would help us prioritize that cluster in the fix deployment.

    Tuesday, March 17, 2015 4:55 AM
  • Sure thing David, and thanks so much!

    bravesemployment.com
    bravesobp.azurewebsites.net

    Tuesday, March 17, 2015 10:19 AM
  • Will I need to restart my website for the fix to take effect?  

    @David, will you post here when it has been deployed?  Is the incident being tracked somewhere by Microsoft?

    (As you can tell, I'm new to this.)  Thanks!

    Tuesday, March 17, 2015 6:58 PM
  • Hi Beth,

    The estimate is a little over 5 hours from now, so around 5:30pm PST. You will not need to restart your site.

    At this point, we are just using this thread to track the incident.

    David

    Tuesday, March 17, 2015 7:21 PM
  • Hi,

    The fix for the issue with ":" in the query string or path has been rolled out for your sites.   We believe we covered all the sites reported in this forum.  Could you verify and let us know if problem persists?

    Thanks! 


    Suwatch

    Tuesday, March 17, 2015 7:34 PM
  • Correction to Suwat's previous comment: this applies to all the sites that were mentioned on this thread before Beth's case. Beth, for your site, we are still looking at the timeframe in my previous mail. Sorry for the confusion.
    Tuesday, March 17, 2015 7:40 PM
  • Thanks for the clarification.  I'll go ahead and go on my bike ride then!  I'll check back in the AM.  Thanks again David and Suwat!
    Tuesday, March 17, 2015 7:42 PM
  • Beth, your site should be fixed now. Thanks for your patience.
    Wednesday, March 18, 2015 12:37 AM
  • Yep, we're back!  The Atlanta Braves Gameday Staff Leadership thanks you very much!  
    Wednesday, March 18, 2015 1:24 AM
  • Correction to Suwat's previous comment: this applies to all the sites that were mentioned on this thread before Beth's case. Beth, for your site, we are still looking at the timeframe in my previous mail. Sorry for the confusion.

    Hello, this is affecting our sites as well. This comment makes it sound like only the sites mentioned in this thread have been fixed. Is there an ETA for a fix for everyone? 

    For us, I was able work around this by using the *.azurewebsites.net URL instead of our domain for the requests in question, but it would be nice to have an official fix for this so these could be switched back to using our domain.

    If you are fixing these by request, I'd love a fix for our domain:

    https://api.essenzasoftware.com

    Thanks.

    Wednesday, March 18, 2015 8:40 PM
  • Hi,

    can anybody confirm if there was a global fix rolled out for this?

    We got the issue on a variety of sites with different subscriptions and in different data centres. Today they all work again.

    However, we temporarily switched access to our main production site to a back-up instance on a VM. Would be nice to get some confirmation about changes, before we switch back...

    Thanks.


    bgx


    • Edited by bgx08 Thursday, March 19, 2015 12:36 PM
    Thursday, March 19, 2015 12:30 PM
  • @bgx08: at this point, the fix is not yet generally deployed, but it will be soon. However, if you no longer see the issue, you can have confidence that it is deployed to your cluster, and that it will not randomly get back into the bad state.
    Thursday, March 19, 2015 4:57 PM
  • Hi David,

    We are still experiencing the same issue to our site with IP SSL. It is possible to know when the fix gonna be rolled out to Southeast Asia region?

    The Url is:

    https://m.bingkis.co.id/mobile/gifts/searchlisting?searchFilter={%22CurrentPage%22:1,%22Keyword%22:%22jungleland%22,%22GiftSearchTableTypes%22:[{%22SearchFieldID%22:%221%22,%22SearchValueID%22:%220%22,%22GroupID%22:%220%22,%22IsGroup%22:%22True%22}]}

    Response Header
    Response    HTTP/1.1 500 URL Rewrite Module Error.  

    Response Body
    <h1>Error 500</h1>
                <p>The Azure Web Site has experienced an unexpected service error. We apologize for the inconvenience. Please try to reload the page or visit it again soon. </p>

                <p>If you are the Administrator of this web site, please visit the <a href="http://www.windowsazure.com/en-us/support/service-dashboard/">Azure Service Dashboard</a> for information about the status of Azure Services.</p>

                <p>    You can also visit the <a href="http://social.msdn.microsoft.com/Forums/windowsazure/en-US/home?forum=windowsazurewebsitespreview">Azure Web Sites forum</a> to discuss this.</p>

    main site is https://bingkis.co.id

    Thanks


    Thursday, March 19, 2015 9:41 PM
  • @ajason: I will look into this now. In the meantime, since you're using a CNAME, you can easily unblock yourself by temporarily switching to SNI SSL mode, per workaround above. Can you give this a try?
    Thursday, March 19, 2015 9:49 PM
  • Thank you for the prompt response. The SSL is pointing to VIP. So it will require us to change the DNS should the VIP changed and also retesting payment gateways integration.
    Thursday, March 19, 2015 10:12 PM
  • @ajason: the issue should be globally fixed now. Please confirm that everything is good with your site.

    Sorry that it took so long!

    Friday, March 20, 2015 8:17 PM
  • David, thanks for the update.

    We did a temporary fix by replacing ":" to "|" pipe, that was working. We will revert it back in the next release.

    Sunday, March 22, 2015 9:59 PM