Consolidating web roles and worker roles dynamically

Question

We have a pool of web role and a pool of worker role. Each worker role does long running batch processing. Worker roles need not be running all the times. We have built some logic around dynamic scaling to minimize COGS but once a worker role is down, bringing it online will require amount of time required by paving Azure machines (~15 minutes) .

Instead, ideally what we would like to do is to have some way so that a role can behave as either web role or worker role. So, let's say there are X number of web roles and Y number of worker roles running and we realize that we need to spawn more worker roles (not threads but new roles),  then we can 'convert' a few of X web roles from web role to worker role. Web role/worker roles are just DLLs that are invoked by different hosts. So in theory that should be possible.

(How) can it be done?

Answer 1

Most folks I know that are doing this simply have the worker processes running as background threads in the web roles. So there's not a seperate role persay Just background processes.. Only trick is to make sure you balance the procssing of the services so that neigher suffers.

Answer 2

It seems like the main challenge is handling the load balancer. In Windows Azure, a role (not individual instances) is set up to receive traffic that comes to a given port. If you want some of the instances of your role to take web traffic and some to not, you need a way to remove certain instances from the load balancer.

I think you can actually do this... but you'll have to set those instances (the ones acting as workers) to the "Busy" state. Take a look at the RoleEnvironment.StatusCheck event. If you implement a handler for that event and respond by calling SetBusy() on the passed in argument, you should be taken off the load balancer for a short period of time. I think if you keep responding this way, you can stay off the load balancer indefinitely.

So what I'm suggesting is having your role be a web role (configured to take traffic), and have the instances manage their participation by setting themselves as Busy when they don't want to take traffic. You'll still need some sort of orchestration so each instance knows which "role" it's supposed to perform. Depending on how you're going to trigger this, it could be as simple as a table with columns "instance ID" and "role."

A downside of this approach is that the role instances that are acting as workers will have a "Busy" state, which makes it hard to tell if they're functioning properly (e.g. in the portal UI).

All of this said, Brent's answer may be better, which is to simply not pursue this strategy. Instead, have every instance act as both.