Why do changes to cscfg take so long?

Question

So I very much appreciate how much faster cloud service deployments are these days. The Azure guys have made huge strides here. But I still find that changes to the cscfg of a running cloud service seem to take much longer than I would expect. For example a simple setting change in my cloud service often takes up to 5 minutes to complete.

We have a few settings (appsettings) that we change periodically to alter the bahavior of our apps and it just seems to take sooo long. I would have thought 30secs to maybe 1 minute at most where it takes 5mins.

Can anyone add any color as to why this is? It seem like the fabric just needs to send the new cscfg out to each instance of each role and apply the change. I'm guessing it does this one instance at a time to make sure they change goes ok, but even so this would seem to occur quickly as the app just needs to restart (and I have pretty simple web apps, web services, and workers).

Is there something I am missing?

Answer 1

Hi,
 
I am trying to involve someone familiar with this topic to further look at this issue. There might be some time delay.
 
Appreciate your patience.

Answer 2

Hi,

I deploy a sample web role and worker role to test the issue. I upgrade the application by uploading a cscfg file. It only takes less than 30 sec to complete. Have you make a test with a  sample web role or worker role?

Also, if you has multiple instances, all instances of each role that is being upgraded and belong to the first upgrade domain are stopped, upgraded, and brought back online. Once they are back online, the instances in the second upgrade domain are stopped, upgraded, and brought back online. So it will take longer time to complete.

If you want to know more details, please check Overview of Updating a Windows Azure Service.

---

Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.

Regards,

Connie Ma

Microsoft Online Community Support

Answer 3

I am definitely not seeing this sort of speed. I have a webrole deployed and as a Cloud Service test I went and changed a simple appsetting using the Azure Preview Portal on the Configure tab of my Cloud Service. From the time I clicked Save until the time the the Portal told me the process was complete took fully 6 minutes.

I have two ExtraSmall instances running this webrole in the US NorthCentral datacenter. THis test was a simple appsetting change, it was not an upgrade or update.

This 6minute time is typical of what I see when changing appsettings in other Cloud Services I have. Why is it this differs from what you are seeing by so much (mine is 12x slower [30sec vs 6min])?

Answer 4

Typically once applied changes are rolled through your Update Domains one at a time - if you were running hundreds of instances I'd have thought that, but at two instances doesn't sound like it should take terribly long, but bear in mind the more update domains, the longer config changes will take.

You can also in your code (and the VS tools with SDK1.3 and earlier did by default) have your instance reboot when a configuration change happens - required in some circumstances, but sounds like it's not what you're looking for.

If your class that inherits from RoleEntryPoint 'Cancels' the Changing event the machine will reboot to apply the change.

This would make six minutes feel about right - sequentially rebooting two VMs plus a little admin overhead.

There's more details on handling configuration changes here: http://leastprivilege.com/2010/12/25/handling-configuration-changes-in-windows-azure-applications/

I'd check the Changing handler first, particularly if its code that was created with an early release of the SDK.

Answer 5

After playing with this some more using the Azure Mgmt API I have found it really only takes about 2 minutes. I am thinking the Azure Portal is somehow taking a lot longer to detect the changes are complete (or at least report them complete). I am happy with 2 minutes.