none
Management portal is down?

    Question

  • Tried to deploy application to Production and faced with Management Portal not loading.

    Sometimes it gives error like

    Sorry, Windows Azure was unable to complete your request. Please refer to the Windows Azure forum where current issues and topics are discussed.

    Sometimes Silverlight portal is loaded but hangs when getting "Hosted Services".

    Are there any troubles with it? When will it be recovered?

    Thanks,

    Sergei.

    Wednesday, February 29, 2012 9:00 AM

Answers

  • Hi Sergei,

    Yes the portal is down, here is the description of the issue:

    1:45 AM UTC We are experiencing an issue with Windows Azure service management. Customers will not be able to carry out service management operations. We are actively investigating this issue and working to resolve it as soon as possible. Further updates will be published to keep you apprised of the situation. We apologize for any inconvenience this causes our customers.

    You can follow any outages on the service dashboard: http://www.windowsazure.com/en-us/support/service-dashboard/  

    Or you can follow @azurestatus on Twitter.

    Sandrino

    • Proposed as answer by Sandrino Di Mattia Wednesday, February 29, 2012 9:08 AM
    • Marked as answer by serg_stone Wednesday, February 29, 2012 9:16 AM
    Wednesday, February 29, 2012 9:08 AM

All replies

  • Yep, down it is. Impossible to login and impossible to deploy from Visual Studio.

    DNS for Windows Azure dnsazure.com

    Wednesday, February 29, 2012 9:03 AM
  • Hi Sergei,

    Yes the portal is down, here is the description of the issue:

    1:45 AM UTC We are experiencing an issue with Windows Azure service management. Customers will not be able to carry out service management operations. We are actively investigating this issue and working to resolve it as soon as possible. Further updates will be published to keep you apprised of the situation. We apologize for any inconvenience this causes our customers.

    You can follow any outages on the service dashboard: http://www.windowsazure.com/en-us/support/service-dashboard/  

    Or you can follow @azurestatus on Twitter.

    Sandrino

    • Proposed as answer by Sandrino Di Mattia Wednesday, February 29, 2012 9:08 AM
    • Marked as answer by serg_stone Wednesday, February 29, 2012 9:16 AM
    Wednesday, February 29, 2012 9:08 AM
  • Hi Sandrino,

    Can you post a link to the "rage-flame-quit" forums ?

    And one about activating Worker threads from a running service ?

    Thank you in advance ... and make them fix that st00pid management portal :)

    Kind regards

    Wixx

    Wednesday, February 29, 2012 9:24 AM
  • This is a problem affecting users worldwide.

    You can the details on the service dashboard.

    Latest Info at 10:30am UCT says the hotfix is being applied but i still cannot use the portal:(

    Wednesday, February 29, 2012 11:45 AM
  • I can not access the dash board. If I go to http://www.microsoft.com/windowsazure/support/status/servicedashboard.aspx I get a generic "Internet Explorer can not display the web page". I've tested from multiple computers in different locations (both Sweden and Germany). All give same result. I've been on the phone with a Microsoft employee who had to read out loud from the web page for me (he was able to access it).

    There is an ogoing Azure issue with incoming traffic to a subset of all hosted services. Could it be that the status page for Azure is hosted in Azure? Which would be... hmm.

    • Edited by Nitramafve Wednesday, February 29, 2012 11:50 AM
    Wednesday, February 29, 2012 11:48 AM
  • From Norway right now I get : "Internet Explorer can not display the web page". When trying to access the Management Portal.

    Also my web applications, deployed on Azure, is not accessible. Giving the same error message.

    Do we have a resolution time on this issue?


    Mcad

    Wednesday, February 29, 2012 12:17 PM
  • Same here...no management portal, no web applications, no website no word when it will be back.

    Some more news would be good.


    Peter (BTMI)

    Wednesday, February 29, 2012 12:23 PM
  • Hey All,

    Or atleast a better place to flame M$ while we wait ...

    I offer a HUG to the most creative M$ or Azure flamer here : 99.9% uptime, but we do not offer Management services or the option of updating your programs.

    Kind regards

    Wicc

    Wednesday, February 29, 2012 12:35 PM
  • This is causing a lot of problems for my company, and I'm really surprised Microsoft hasn't solved it yet.

    Not that it helps fix the issue but I found this ZDNet article informative:

    Microsoft grapples with Windows Azure outage

    Also if Microsoft is going to post status updates on their Service Dashboard then they should make them accurate, or at the least, reasonably up to date. Telling your users the service is running when it isn't doesn't help matters.


    • Edited by Bernie_ Wednesday, February 29, 2012 12:49 PM Fixed link
    Wednesday, February 29, 2012 12:48 PM
  • Portal is up here (Nicaragua-connecting to US-South Central), although, it doesn't seems to be working completely ok, it keeps trying to refresh on some of the services we have in one of the subscriptions.

    Regards.

    Wednesday, February 29, 2012 1:24 PM
  • I can also see the Management Portal...all looks fine there, but the websites are still inaccessible.

    Peter (BTMI)

    Wednesday, February 29, 2012 1:33 PM
  • Management portal - Down.

    Web applications - Down.

    Database access and deployed webservices seems to be running fine. We have it all hosted in North Europe domain.

    This is a MAJOR problem for our organization, will we get compensated for this? Microsoft has a guaranteed 99.9% uptime promise for Azure - which is clearly breached here!


    Mcad

    Wednesday, February 29, 2012 1:40 PM
  • Mcad, it's a serious problem to our organization as well and I would also like some compensation, but keep in mind that a 99.9% uptime promize (per what, month, year? rolling or between specific dates? can't remmeber) means like 8.7 hours of down time per year. As far as I've seen myself, our applications have only been completetly unavailable for ~3 hours.


    Nitramafve

    Wednesday, February 29, 2012 1:44 PM
  • Dear Customers,

    Your webservices have been avaiable the entire time. Only the management portal fell like the Gates of Troy. So this downtime is not part of our graranteed 99.9% services nomatter. We can even shut it down for days and you will just need to remember:

    This... is... SPARTA!

    Kind regards

    Wicc

    Wednesday, February 29, 2012 1:50 PM
  • I can now access the management portal, but...

    I just did a new VM role deploy and the deploy is freezed on setting up windows for first use....

    I'm 100% sure this problem is not related with my project or with my VHD. I used same project yesterday and the same VHD image that was already uploaded.

    Wednesday, February 29, 2012 2:46 PM
  • Still Major problems in; Windows Azure Compute - North Europe

    Feb 29 2012 2:30PM We have determined that about 37% of hosted services in this
    sub-region are impacted by this issue. The restoration steps to mitigate the
    issue are underway. Further updates will be published to keep you apprised of
    the situation. We apologize for any inconvenience this causes our customers.

    My web applications and management portal are still down, my customers have been without critical information for about 4 hours now :(


    Mcad

    Wednesday, February 29, 2012 2:49 PM
  • I got report one of my web site is not accessible this morning. So I tried to re-deploy my web site, but the management portal is not working either.

    This is very critical for our business. Please fix it ASAP.

    Thanks,

    Josh

    Wednesday, February 29, 2012 3:00 PM
  • So, do anyone has an update about the estimates, when it will be fixed ? My main services are in status: "Transitioning..." and doesn't answer by *.cloudapp.net url for last 6-7 hours.

    What has happened ?

    Wednesday, February 29, 2012 4:59 PM
  • I was able to start a deploy using a package on the portal about 4 hours ago, and the deploy is still not finished...

    Wednesday, February 29, 2012 5:23 PM
  • Hurray! my services are available! but Management portal is not answering from time to time :( 
    Wednesday, February 29, 2012 6:00 PM
  • We missed today's deployment window for Production. We will try tomorrow. Management portal is running but Production sites unavailable. That's not good.

    Wednesday, February 29, 2012 6:45 PM
  • Same happened to me in one of our subscriptions, deployment doesn't finalize yet. But I could sucessfully deploy in another subscription. Both subscriptions are in South-Central US.
    Wednesday, February 29, 2012 7:44 PM
  • That concerns me, because I'm testing Azure for a LOB app and having 6~7 hours with the app offline is not possible.
    Wednesday, February 29, 2012 9:19 PM
  • Hi Diego,

    I understand your concern.. we are about to launch an app for a marketing campaign to all of Germany (18 million brochures) for our client, the timing of this outage is not good.

    A few things surprise me..

    - only 22 replies and 600+ views of this topic until now? How many Azure users are there?

    - how could such a single point of failure regarding the Management services have crept into the design?

    - why isn't MS communicating much clearer about this? This is a Major credibility/reputation issue that will hurt Azure for years to come.. No tweets from Scott Guthrie either.

    - I for one would like to understand what is happening, or at least be able to say something meaningful to my stakeholders. How can a faulty certificate bring the entire system down? Pretty hard to wrap your head around..

    - There isn't one capable/empowered communications advisor among Microsoft's 90000 employees?

    Baffled Lex is baffled.

    Wednesday, February 29, 2012 9:36 PM
  • This is not acceptable. We are loosing customers on this issue.... We have chosen Windows Azure due to it's redundancy, and not this, and entire day og of service.... It's really bad.... No official statement.... The ironic part is that, if we had gone with a self hosted solution it would max have been a 2 hour down time..... Hello Microsoft..... Are there no internal procedures to deal with this thing ? Maybe the Azure platform was launched 2 years to early..... Are we paying beta testers :-( This is a catastrophe..... 
    Wednesday, February 29, 2012 9:57 PM
  • All of my sites are back now but I dare not touch anything related to the Management Portal until there are green ticks all the way down.  It is strange that only a percentage of customers were affected in each region, and why am I always one of them? 

    Nothing raises the anxiety levels more than a customer telling you your website is down.  My only positive from this experience is that the "safety in numbers" rule applies conversely here, because there is nothing we can do about it, fixing it is a high priority and with it being widely reported in the press, it is Azure and not our reputation as content providers that is in the firing line.

    It would be nice to find out about these problems first and not second though.  I hope the Microsoft conducts a serious review of this event and shares with us the results.


    Peter (BTMI)

    Wednesday, February 29, 2012 10:51 PM
  • Lots of over dramatization on this thread.  It's not like your Azure hosted services are down, we just can't deploy.  Trust me, this is majorly affecting our deployment plans for an end of the month change over and a possible loss of service because this is a NEW hosted deployment for release tomorrow morning.  However, to act like everyone is losing customers because of this is silly.  Most everything Azure related is up and running, and we are only missing the ability to modify through management portal.  Hence the reason this thread has so few posts and views.  People don't go into management portal very often, in fact, yesterday was the first time we needed to 'modify' anything in over 5 months.

    Having said all that, Microsoft should be offering service credits for this.  I can understand a 1-2 hour outage, but this is dragging on all day and it is becoming problematic for those of us who need to manage their services.

    Wednesday, February 29, 2012 10:54 PM
  • You were lucky then not to be in the percentage of users that also lost their web applications because of related problems with their Windows Azure Compute services which meant dead links for half a day which is the cause of much of the hand-wringing here.

    Peter (BTMI)

    Wednesday, February 29, 2012 11:05 PM
  • NO, you are wrong, services ARE DOWN.  I've hard parts of my services down for over 12 hours now.  I am pissed off beyond belief, and Microsoft has been absolutely no help, except to say, we're working on it.

    At 4AM today (MST), most of my services were down.  Now all but about 3-4 (out of roughly 30) are back up.  But there is absolutely no excuse for this.  None.  I will now consider moving to EC2.  I've had it.  They had a big outage in October timeframe with Storage being down in South Central US, and now this, a worldwide outage of various systems.  The Portal for sure, Some compute, Caching, some SQL problems.

    Mark

    Wednesday, February 29, 2012 11:15 PM
  • Yeah, no kidding.  I'm so pissed.  They originally said it was all due to a bad certificate that was issued.  If that WAS the case, why not revoke the cert and replace it.  This supposedly happend at 5 PM PST on Feb 28th!  Almost 24 hours ago.  If that's really the cause, why all this, why have to do some hotfix to every single instance of Azure in the world ?  They are NOT telling us the truth.
    Wednesday, February 29, 2012 11:16 PM
  • Our web site http://www.mazily.com/ has been down all day since lunch time (Sweden). Still not up. Very disappointed by the information I can find at the service portal. Seems that Microsoft is not revealing the entire story here or very delayed... The service dashboard is also not very user friendly or informative in my eyes. Especially if you are not a technician. Shape up!
    Wednesday, February 29, 2012 11:27 PM
  • My compute nodes are in fact down.  My main hope is that after this issue they will be a lot more careful in ensuring that they put more focus on making sure something like this does not happen again.
    Thursday, March 01, 2012 12:26 AM
  • Our production deployments and production roles are degraded, our production instances unhealthy. Our deployed applications and web services can't access our SQL Azure database.

    SQL Azure Data Sync - East Asia - Service Unavailable = :'(

    It's been like this for over 16 hours. I went to sleep last night hoping things would get better but they've only gotten worse.

    All day we've been hammered with calls from our clients and business partners wondering why they can't see their published content, why they can't log into their applications, why stuff is down. All we can tell them is, "Microsoft's Azure is down." 

    I'm curious what kind of compensation we'll get for Microsoft failing to meet the guaranteed uptime. 

    Thursday, March 01, 2012 3:29 AM
  • Services are indeed down. My app website was down the whole day and there is no more info than we're working on it. My ISP does that, but this was the last thing I expected from Windows Azure services.

    When I sell Windows Azure to my customers I always emphasize the service reliability, but now what?? If there are people that wasn't affected by this problem consider yourself lucky and stop telling us that we're overreacting, cause we're not.


    Thursday, March 01, 2012 3:53 AM
  • I've been using Azure since beta and this is incredibly disappointing and has been damaging to our business.  Not only were my company's systems down for over 9 hours yesterday but I have a critical update to deploy for 1st March and I can't deploy the updates.

    The Azure Service Dashboard is currently reporting that the Windows Azure Platform Management Portal [Worldwide] has a status of "Service is running normally" and i can tell you it isn't!  So the updates are misleading (or possibly very slow in being fed through) - which has pretty much been the case since this started yesterday.  Come on Microsoft.

    I also have a separate staging deployment that failed when the service went down yesterday and I can't delete, upgrade or start it.  Something else to look at.

    • Edited by Bernie_ Thursday, March 01, 2012 5:45 AM Removed unecessary text
    Thursday, March 01, 2012 5:42 AM
  • Then I am trying to update Deployment it say me: An Internal Server Error occurred. Please retry your last operation at a later date.

    Bernie_ do you get the same problem ? 

    Thursday, March 01, 2012 6:32 AM
  • @SuperPups: you are wrong, OD-Mark is correct. At least, for our situation and location (West Europe). Our applications were down too. Web Role instances couldn't communicate to database because Access Control was down.  And for a long time these services were still marked as unavailable in the Management Portal (once it was up again): Database, Data Sync, Reporting, and "Service Bus, Access Control and Caching".

    Thursday, March 01, 2012 8:18 AM
  • Hi MaximKur

    I just tried the management portal again, as of 8:45am GMT, and I can now delete a faulty staging deployment and am in the process of uploading a new staging deployment.  It's at 40% so far and uploading very, very slowly so I'm keeping my fingers crossed it will work. 

    I'll believe it when it happens though because at this stage I don't quite trust the service dashboard.

    Just so you know I'm targetting the Northern Europe Datacenter.

    Good luck.

    PS. This blog posted a few hours ago might shed a "bit" of light on the issue: "Windows Azure Service Disruption Update" by Bill Laing, Corporate VP Server and Cloud

    Thursday, March 01, 2012 9:06 AM
  • So, a leap year bug! How embarrasing is that..

    Our webapplications was down for over 12 hours in the North European datacenter.


    Mcad

    Thursday, March 01, 2012 9:17 AM
  • Extremely... I think we could have lost 2 prospects who are testing our system currently... But I can't imagine the damage this has done to companies with large scale customers.

    I mean we have chosen Windows Azure due to the redundancy..... How can we explain this to our customers :-S

    Thursday, March 01, 2012 10:21 AM
  • If this is of any help I've been attempting to deploy for the last two hours (since 8.30am GMT) and this is what I've found:

    • If I deploy to staging, and have 3 web roles the 3rd one won't start up - if I try rebooting the problematic 3rd instance, or reducing the instance count down to 2, nothing happens.  Because it won't start up I can't Swap VIPs
    • I can deploy straight over the top of a production deployment which works successfully - unforunately it has a temporary performance impact
    • I've found that the online management portal isn't working correctly so have been using Cerebrata's Cloud Storage Studio 2 with more success

    Good luck

    Thursday, March 01, 2012 10:27 AM
  • I still can't deploy any sites :( it always says me: An Internal Server Error occurred. Please retry your last operation at a later date.
    Thursday, March 01, 2012 12:30 PM
  • Hi MaximKur

    Not sure if this helps but I had to completely delete production and/or staging deployments so that I could get a clean deployment done that worked.  Any existing deployments that were in place before this problem arose yesterday (29th Feb) were problematic to swap around or adjust in any way. 

    I also had to use a third party tool (Cloud Storage Studio 2) to interface with Azure.

    Good luck.

    Thursday, March 01, 2012 1:18 PM
  • My final couple deployments started working late last night, around 7PM PST.  Everything seems to be running normally now.

    We had our CEO and a SVP at Mobile World Congress selling our software yesterday.  They literally just hid so as to not have to explain why they couldn't demo our software.

    Luckily for me, our most used production system was not affected...somehow.  

    Now it will be interesting to see what Microsoft does for us, and for them selves.  I personally am going to need some assurances they will take actions to make sure this doesn't happen in the future.  I'm already getting pressure that we should just move to EC2.  However, I see they had a 4 day outage on storage on EC2 last April.  Sigh.  I guess to some extent this is one of the dangers of going with the cloud.

    Thursday, March 01, 2012 3:08 PM