locked
Basic scalability question in asp.net applications RRS feed

  • Question

  • I have a basic question with respect to multi-tier architecture.

    Say I have an asp.net application with 2 tiers -

    1. Web  

    2. Business + DB access layer

    If a page request takes a long time to execute due to an expensive computation in business layer or an expensive query, how does the multi-tier architecture help me in increasing scalability?

    Because the the asp.net thread (in the web layer) that calls the business layer is still waiting for the response right?

    If this is the case how can the web layer service more requests (as all threads are waiting for response from biz layer)?

    Asynchronous request into the business layer is not possible as the user is waiting for the results to show up.

    Can somebody please throw some light on this?

     

    Monday, November 24, 2008 1:19 PM

All replies

  • Multi-tier solutions are different from high-scalable solutions.  High Scalable solutions are your Youtubes, googles, amazons and ebays.  They offload the “Expensive computation/query” on multiple machines (much like Grid) and aggregate the results.  Also, to be noted high scalable solution architectures are moving away from using database in traditional sense and mostly as persistence data store.  They use distributed ORM cache (NCache, Velocity, Terracotta, Encache, etc.,) for data management.  You may want to think about what is the actual bottleneck in your application and try to solve specifically for it rather than general approach.

    Here is a link to some general high scalable solution approach.

    { Gaja; }

    http://gajakannan.com/netarch.aspx

     

    Monday, November 24, 2008 10:42 PM
  • Then why do people deploy layers on different machines?

    My question is particular to scalability of asp.net applications.

    I have read in books and on the internet that you increase scalability by deploying different layers on different servers. It kinda makes sense when we think in terms of processor usage. But the basic question remains. How does this increase scalability as the threads that have to service new requests are waiting for response from the other layers?

    Tuesday, November 25, 2008 5:40 AM
  • Hi,

    As in previous reply The scalability and layered approach are different , The layered approch might help you little bit of scalability like if you split up your layer across servers  you can utilize the system resources in different servers as well as it is running on different process. But to improve scalability factor we have to think about different ways rather than layered approach. My suggestions are.

    1. Asynchronous call , Even though you told it will not work build the logic like load all the request parallel and wait the main call till all the parallel thread complete.

    2. Use  better DB , and try to optimize your fetching logic.

    3. Offline process the data in DB.

    4.Pre -Process the data and keep it ready to fetch.

    keep trying and pleas  update me ...

    Regards,
    Pramod
    Tuesday, November 25, 2008 6:35 AM
  • The problem you are describing (processing which takes a long time) is not actually a scalability problem (at least not the way Microsoft often describes scalability), which is scaling the number of users/requests to a given system.


    The multi-tier solution that is described is most relevant for improving the scalability of a 2-tier app. Instead of each rich client opening a connection to the DB, introduce a middle tier which does connection pooling to the database. Most of this isn't really relevant for web apps.

    I've blogged about various ways in which you can solve the problems you've described in this post:

    Scaling long-running web services


    Hope that helps.
    Tuesday, November 25, 2008 7:13 PM
  • Udi, I read your blog and here is what I infer (I believe thats what you attempted to mention in your sequence diagrams).  Web service is just a entry point for a long-running service, and the actual operation is to off loaded to a queue/workflow/BPM type solution.  When you offload to an external process then you need either polling or async callback to check completion status. 

    But the original post for the request did not want to create a "async callback" or keep the connection alive (which is polling).  Hence my recommendation was to scale the actual bottlenect using a "scalabale architecture".  We would not be using Google or eBay if the request we put goes into a callback or clocking for completion right?

     

    { Gaja; }

    http://gajakannan.com/netarch.aspx

    Tuesday, November 25, 2008 8:18 PM
  • Thanks for that great blog post Udi. I was actually considering using MSMQ and your post has been very useful.

    The problem that I am referring here is very real. Scalability and throughput are different. You can't help throughput in some cases. I mean if a long running operation is going to take 20 minutes, it is going to take 20 minutes. There is precious little you can do about it. (Ex: read and parse a 20 mb file and give the report to the user or a b2b client app - the external b2b client insists on waiting and i cannot do anything about it).

    You can get a buy in on time taken to process a request (throughput) but never on denial of service (scalability). Most people deal with this problem by throwing hardware at it. But I wanted to see if people are doing anything to keep the "request handling threads of asp.net free".  I found some answers in aync pages and the related stuff but that doesn't solve all the problems for me.

    If people are doing more stuff or if you guys are using async patterns with lot of success, please share.

     

    Wednesday, November 26, 2008 6:51 AM
  • I have used Async many times in my past.  Truth is it works as expected.  Things you need to consider in real world scenarios are good exception handling,

    • client application terminated before you can call back
    • your application is recycled during the middle of processing
    • client application terminated and restarted, so from client's end they are back but can not be called back.

    Udi's recommendation in sequence diagram to generate a GUID for that request and using that GUID for during callback would help mitigate those cases in your application (this means your client app also has to implement the code to follow this model).

     

    Remember my response was with respect to no use of queueing or Async.  If you have flexibility to implement option then you should use Async.

     

    Below is a link from msdn on Async best pratices (for Event driven, but this would suit your needs as well)

     

    { Gaja; }

    http://gajakannan.com/netarch.aspx

     

     

    Wednesday, November 26, 2008 6:03 PM
  • GajaKannan,

    > But the original post for the request did not want to create a "async callback" or keep the connection alive (which is polling).

    The underlying behavior of asp.net threads blocking while the request was being processed caused the scalability problem. You cannot solve one without the other.

    >
    We would not be using Google or eBay if the request we put goes into a callback or clocking for completion right?

    An asynchronous architecture isn't necessarily visible to the user. BTW, both Google and EBay make use of these patterns.
    Wednesday, December 3, 2008 1:02 PM
  • Udi,

     

     Udi Dahan The Software Simplist wrote:


    > But the original post for the request did not want to create a "async callback" or keep the connection alive (which is polling).

    The underlying behavior of asp.net threads blocking while the request was being processed caused the scalability problem. You cannot solve one without the other.

     

    We could solve this problem by nailing down the longest running process and parallelizing the threads at the server to get the results faster.  ESB is certainly one piece of the puzzle.  But just putting a ESB is not going to improve the application performance. 

     

     Udi Dahan The Software Simplist wrote:


    > We would not be using Google or eBay if the request we put goes into a callback or clocking for completion right?


    An asynchronous architecture isn't necessarily visible to the user. BTW, both Google and EBay make use of these patterns.

     

    Yep, I know async can be made invisible to user, like using ajax in web apps, if the user experience has built to engage the user on some other activity and bring the notification back when the requested data is received.  But hey, that is a typical implementation of Async, otherwise we would call it as Synchronous right or Async implemented very wrongly.

     

    { Gaja; }

    http://gajakannan.com/netarch.aspx

    Friday, December 5, 2008 9:27 PM
  •  GajaKannan wrote:

     We could solve this problem by nailing down the longest running process and parallelizing the threads at the server to get the results faster.  ESB is certainly one piece of the puzzle.  But just putting a ESB is not going to improve the application performance. 

     


    Taking a sequential piece of business logic and making it run in parallel, correctly, getting lower latency, while not hurting throughput is not a trivial task, to say the least.


    Actually, for many DB bound operations, increasing the number of threads only works to increase contention, causes deadlocks, and often hurts performance as well as throughput. That's not to say that it isn't possible, but that it requires revisiting (aka re-designing and re-writing) considerable amounts of code, DB schemas, etc.


    A non-blocking style of interaction can free up the ASP.NET bottleneck improving throughput (scalability) with minimal changes to the code.

    Monday, December 8, 2008 7:51 AM
  •  Udi Dahan The Software Simplist wrote:

    Taking a sequential piece of business logic and making it run in parallel, correctly, getting lower latency, while not hurting throughput is not a trivial task, to say the least.


    Actually, for many DB bound operations, increasing the number of threads only works to increase contention, causes deadlocks, and often hurts performance as well as throughput. That's not to say that it isn't possible, but that it requires revisiting (aka re-designing and re-writing) considerable amounts of code, DB schemas, etc.


    A non-blocking style of interaction can free up the ASP.NET bottleneck improving throughput (scalability) with minimal changes to the code.

     

    Can you spell out what is your implementation for non-blocking style of interation?  Async process?  If the original post left room for async, that would be my first choice.  I am not suggesting Parallelizing and HPC are easy and trivial, but given the requirements that sounds better option than giving the poster exactly what he did not want.  I think thats where we disagree.

    Monday, December 8, 2008 3:54 PM
  • Non-blocking is similar to async, but slightly different. While you can perform a call on a different thread using an async call, if you're first thread has nothing else it can do, the overall interaction is still blocking. A non-blocking style of interaction will probably be implemented with a one-way messaging framework.

     

     

     

    Wednesday, December 10, 2008 6:35 AM
  • Udi,

    one-way messaging framework, will work in fire-and-forget type of interaction.  But in the case as explained in the original post needed a feedback when the work is completed.  So how would that happen?  Would the caller/browser poll the server at regular intervals?  BTW, in the async process, if the first thread is still waiting for the call to and not doing anything else, then it is NOT ASYNC.

     

    Can you explain with a fictionary example of this non-blocking style that works differently than async?

     

    Not that wikipedia is always correct, end of the day it is collective knowledge created by every day folks like you and me, but this is what I found in wikipedia for asynchronous...

     

    In programming, asynchronous events are those occurring independently of the main program flow. Asynchronous actions are actions executed in a non-blocking scheme, allowing the main program flow to continue processing.

    Wednesday, December 10, 2008 3:29 PM
  • Gaja/Udi,

                   One of the solution at least conceptually is to return a response to the client that will make it poll continously like Udi has suggested in his blog. Please share any implementation/sample you might have.

    Also I was thinking along IIS worker process isolation mode. Basically have a different worker process service another web service and offload some of the processing. This is because another worker process will have its own pool of threads. I know this might cause OS thrashing due to frequent context switches by the processor but with such powerful processors out there we can give it a try. I think there might just be a solution which is the combination of Queueing, some way to keep the connection alive or make the client poll without it realizing and IIS worker process isolation mode. The end result would be to achieve maximum scalability with minimum hardware. I really do not care too much about processing time of the request (for this app). I care about not getting the "server busy".

    Keep em coming.

     

    Friday, December 12, 2008 7:27 PM
  • Udi's blog is a good place for the information you are seeking for.  You can also look at .NET Stocktrader application that is built to demonstrate SOA.  You will find anything and everything you may need, except stock trader does not explicitly use a ESB.  By adding ESB like nServiceBus or Mule, you are extending the life of your application to a better manageable solution.
    Saturday, December 13, 2008 4:41 AM