locked
Advice on webform architecture RRS feed

  • Question

  • User-1862720597 posted

     Hi,

    First of all...I hope I have the right forum.

    I have a webform for an ecommerce website that can process hundreds, maybe thousands of purchases at specified times of the month.

    Users will sign up for autoshipments once a month and I will charge them for their order and submit their order to have their products delivered to them.

    This is the flow:
    -I go through everyone in the database that has signed up for autoshipments
    -I get their order that they have specified that that they want to recieve every month
    -I submit the order to the Payment Processor and charge them for their order
    -If Successful, I write their order to my database
    -I send an email to the distributor (warehouse) so they can ship their order
    -Send an email to the user informing them that their Autoship order has been processed

    ...and then I do this for every user in the database. I've tested this with about 5 users and the process takes about 15 seconds. If I have 1000 users, this will take a considerable amount of time and this will use up a lot of resources.

    Is there a better way of doing this so that my website doesn't go down everytime I run this process?

    Thanks

    Monday, May 17, 2010 10:16 AM

Answers

  • User-952121411 posted

    Can this be done with a Web Service as well? I'm wondering because my website will be hosted by a web hosting company and I'm not sure if they will allow me to install a Windows Service on their server.
     

    A web service by nature is stateless as well so it does not have the ability to run a single process indefinitely with a timer, etc like a Windows Service.  Take a look to the following link where a similar question was asked: http://forums.asp.net/p/1268158/2393225.aspx

    Since you are having your web app hosted, you could use your main ASP.NET app or another ASP.NET app to contain timers that start up in one of the Session_Start or Application_Start events to mimic such a process.  I always found this difficult to perfect for periodic processing because of all of the intricacies of ASP.NET being hosted by IIS like the worker process being recycled, session timeout, etc.  However there are others on this forum that have claimed to have done quite well with running a long running task in an ASP.NET web app, most often for a similar reason (hosted app where Windows Service can not be installed).  You may want to look to the following for more information:

    Simulate a Windows Service using ASP.NET to run scheduled jobs:

    http://www.codeproject.com/KB/aspnet/ASPNETService.aspx

    Hope this helps! Smile

     

    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Wednesday, May 19, 2010 8:46 AM

All replies

  • User-952121411 posted

    First of all...I hope I have the right forum.
     

    Sure you do! Laughing

    Users will sign up for autoshipments once a month and I will charge them for their order and submit their order to have their products delivered to them.

    The portion above is obviously appropriate to be served up through your ASP.NET web app.  Now let's dissect the rest of the processing:

    -I go through everyone in the database that has signed up for autoshipments
    -I get their order that they have specified that that they want to receive every month
    -I submit the order to the Payment Processor and charge them for their order
    -If Successful, I write their order to my database
    -I send an email to the distributor (warehouse) so they can ship their order
    -Send an email to the user informing them that their Autoship order has been processed

    You have (2) decent options that I see for doing this is a more asynchronous manner (architecturally speaking, not physical multi-threading speak at this point).  You don't need your ASP.NET application to process all of the above details; it can run alone on its own in parallel.  That indeed places a lot of undue processing on a front end application for processing that is mostly considered background processes.  The 1st option is to overall a lot of the flow and migrate it to a robust tool like Windows Workflow.  While I wouldn't immediatly run there it is a nice option if you see the products and distribution flow scaling out and becoming a much more involved process.  It has a semi-steep learning curve but the trade off is some robust flow functionality.  Check out the following MSDN launch page if you would like more information:

    Windows Workflow Foundation...

    http://msdn.microsoft.com/en-us/netframework/aa663328.aspx

    A more reasonable solution would be to port all of those background tasks into a Windows Service that runs on an interval you dictate to 'wake up' and do all of the processing.  From the service you can call the database, run .NET code, call a WCF or .asmx service, etc. to do any and all of the processing you need.  Since this runs in the background, you will not have the processing load directly in IIS on your web app that you may be incurring currently.  You can also install the Windows Service on another server all together if needed based on your infrastructure setup.  For more information in how to use a Windows Service to process you background tasks, take a look to the following:

    Running a Periodic Process in .NET using a Windows Service:

    http://allen-conway-dotnet.blogspot.com/2009/12/running-periodic-process-in-net-using.html

    Hope this helps! Smile

    Monday, May 17, 2010 3:36 PM
  • User-1916342745 posted

    That sounds like a painfully slow process. I take it that the biggest single delay point is the Payment Processor. 

    I'd be tempted to look at some form of message queue... maybe  ms message queue or mass transit (which I haven't used myself)..  http://devlicio.us/blogs/tim_barcz/archive/2008/10/13/mass-transit-part-1-of-n.aspx

    You' do the database search in the normal way, but write entries to the message queue.  You'd then write a secondary process to subscribe to the queue, and send off requests to the payment provider when items exist. 

    I'd also check there is a way to interact with your payment provider in bulk. 





    Monday, May 17, 2010 3:37 PM
  • User-952121411 posted

    I'd also check there is a way to interact with your payment provider in bulk. 
     

    Most of the Payment processors or gateways I have seen do not expose this functionality (doesn't mean it does not exists - just not in any I have worked with).  One difficulty would be if an individual transaction is declined there may be different business rules that need to be followed based on the requirements.  This would need to be handled separately on a per transaction basis.  Since many times these API's send back transaction results as key value pairs, they tend to be a 1:1 processing nature as robust objects, .NET ADO types, arrays, etc that could hold multiple responses don't seem to be supported often.

    Regardless, porting out the entire workflow to a background process or services will have significantly better performance than trying to jam it all into the single ASP.NET app.  That should make a significant difference to the performance and overall usability when re-architected like suggested.

     

    Monday, May 17, 2010 4:32 PM
  • User-1862720597 posted

    Hi atconway,

    Thanks for your reply. I'm looking at your Windows Service suggestion. Can this be done with a Web Service as well? I'm wondering because my website will be hosted by a web hosting company and I'm not sure if they will allow me to install a Windows Service on their server.

    Worth looking into though!

    Thanks

    Tuesday, May 18, 2010 9:08 PM
  • User-952121411 posted

    Can this be done with a Web Service as well? I'm wondering because my website will be hosted by a web hosting company and I'm not sure if they will allow me to install a Windows Service on their server.
     

    A web service by nature is stateless as well so it does not have the ability to run a single process indefinitely with a timer, etc like a Windows Service.  Take a look to the following link where a similar question was asked: http://forums.asp.net/p/1268158/2393225.aspx

    Since you are having your web app hosted, you could use your main ASP.NET app or another ASP.NET app to contain timers that start up in one of the Session_Start or Application_Start events to mimic such a process.  I always found this difficult to perfect for periodic processing because of all of the intricacies of ASP.NET being hosted by IIS like the worker process being recycled, session timeout, etc.  However there are others on this forum that have claimed to have done quite well with running a long running task in an ASP.NET web app, most often for a similar reason (hosted app where Windows Service can not be installed).  You may want to look to the following for more information:

    Simulate a Windows Service using ASP.NET to run scheduled jobs:

    http://www.codeproject.com/KB/aspnet/ASPNETService.aspx

    Hope this helps! Smile

     

    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Wednesday, May 19, 2010 8:46 AM
  • User-1862720597 posted

    Thanks for the article atconway...looks like something I can use.

     

    Wednesday, May 19, 2010 9:53 PM