locked
Handling Long running task in Asp.net ??? RRS feed

  • Question

  • User370196378 posted

    Hi , <o:p></o:p>

    i am working on Asp.net application with database sqlserver.<o:p></o:p>

    I have a long running task which takes 50 minutes processing time now in this scenario user cannot wait for this much of time and my web page will also gets expired.<o:p></o:p>

    basically of long running task does data shifting from one database to another database. In my application multi users can works at the same time this means that users can put multiple requests at the same time.<o:p></o:p>

    Now my problem is that i want to complete this long running task totally offline which would trigger when user put any request, but what happened when another user put a request at same time this means first request would override by second one.<o:p></o:p>

    FOR EXAMPLE GOOGLE  mail importing task.<o:p></o:p>

    if you seen google mail importing task . google provides import your mail from yahoo or others servers when you click on import button then an import task is started and show importing. this importing task runs for one or two days which can be stop by users at any point of time. <o:p></o:p>

    this indicates that importing task is totally offline. <o:p></o:p>

    I want to implement same functionality in my application. please suggest any way to implement above scenario.<o:p></o:p>

    Every suggestion would be appreciated. <o:p></o:p>

    Thursday, March 11, 2010 12:18 AM

All replies

  • User-1636183269 posted

    I will prefer to such huge task by scheduler, and once complete then send mail.

    Thursday, March 11, 2010 1:21 AM
  • User665034424 posted

    You need to use asynchronous calls using a timer in the UI. Bascially the task will be running in the webserver, at each interval query and display something like a progress bar in the page. Even after sometime when the user visit the page again you will be displaying the progress bar based on how much task got completed.

    Thursday, March 11, 2010 1:23 AM
  • User370196378 posted

    As i have mentioned in my post that schedule task can be run for a single users .

    We cannot handle multiple users request using schedules task. 

    I hope you have perspective of Google Mail imports functionality.


    Please suggest if you any another idea.

    Thursday, March 11, 2010 2:54 AM
  • User665034424 posted

    How about doing the task in a multi threaded windows service.

    Let the service update the progress status in db and the UI fetches that info and display it.

    Thursday, March 11, 2010 3:45 AM
  • User-1184423958 posted

    I would suggest to have a flag in Database which helps to identify the completion status of long running task.

    Then, use one of the ASPX page to make request for long running process using BackgroundWorker class. This executes the operation on a separate thread. The RunWorkerCompleted event of BackgroundWorker class occurs when the background operation has completed. In this event, update the completion status.

    Also, i suggest to read the below link
    http://www.codeproject.com/KB/web-cache/NETBackgroundWorker.aspx

    Thursday, March 11, 2010 3:55 AM
  • User370196378 posted

    Thanks for reply

    I have more then one million users and all users  submit their request on daily basis It means that my application

    receives thousands of long running request daily so database flag approach does not solve this problem because

    if i put requests in queue then last request will start after ten days. 


    Please anybody suggest me any other way to do it.

    Thursday, March 11, 2010 8:03 AM
  • User-319574463 posted

    Please anybody suggest me any other way to do it

    You need to run a number of parallel threads in your web service to cope with the volume. You may even need to move that aspect of the processing off to a separate server.

    Instead of thinking about a database flag, you obviously need a table within the database in order to queue the requests.

    Wednesday, March 24, 2010 5:44 PM
  • User-952121411 posted

    Now my problem is that i want to complete this long running task totally offline which would trigger when user put any request, but what happened when another user put a request at same time this means first request would override by second one.
     

    You definitely need to offload this processing outside the main ASP.NET application.  The ASP.NET app will obviously not handle these long running types of tasks well from a client;s perspective and it doesn't make sense for that processing power to have to come from the web app itself.

    When the client requests to initiate one of these long running tasks, all I would have the ASP.NET web app do it update a flag in the database with associated user info obviously for the task that is outstanding.  Then I would create a Windows Service that polls on an interval to the outstanding tasks in the database, and processes them as needed.  You could have the polling done on as little or long time as needed (i.e. every second if you want).  From the Windows Service you can call any service, component, database, layer, code, etc needed to complete the processing.  It does not all have to be done from inside the Windows Service, as its main function is to poll the database and initiate processing.  Take a look to the following for some more information on how to accomplish this with a Windows Service:

    Running a Periodic Process in .NET using a Windows Service:

    http://allen-conway-dotnet.blogspot.com/2009/12/running-periodic-process-in-net-using.html

    Now for performance and scalability (2) other distinct options present themselves.  Multithreading and hardware.  If you have the right type of processors (hardware) on the server where the Windows Service is installed, you could launch multiple processes at the same time if upon polling you find there are > 1 task to be completed.  I would start with just synchronous processing via the Windows Service polling and only scale to a multi threaded solution if you find that tasks are not getting completed fast enough, and the server had processing power on deck to use.

    Hope this helps! Smile

    Wednesday, May 19, 2010 8:58 AM
  • User1137074964 posted

    Now my problem is that i want to complete this long running task totally offline which would trigger when user put any request, but what happened when another user put a request at same time this means first request would override by second one.
     

    You definitely need to offload this processing outside the main ASP.NET application.  The ASP.NET app will obviously not handle these long running types of tasks well from a client;s perspective and it doesn't make sense for that processing power to have to come from the web app itself.

    When the client requests to initiate one of these long running tasks, all I would have the ASP.NET web app do it update a flag in the database with associated user info obviously for the task that is outstanding.  Then I would create a Windows Service that polls on an interval to the outstanding tasks in the database, and processes them as needed.  You could have the polling done on as little or long time as needed (i.e. every second if you want).  From the Windows Service you can call any service, component, database, layer, code, etc needed to complete the processing.  It does not all have to be done from inside the Windows Service, as its main function is to poll the database and initiate processing.  Take a look to the following for some more information on how to accomplish this with a Windows Service:

    Running a Periodic Process in .NET using a Windows Service:

    http://allen-conway-dotnet.blogspot.com/2009/12/running-periodic-process-in-net-using.html

    Now for performance and scalability (2) other distinct options present themselves.  Multithreading and hardware.  If you have the right type of processors (hardware) on the server where the Windows Service is installed, you could launch multiple processes at the same time if upon polling you find there are > 1 task to be completed.  I would start with just synchronous processing via the Windows Service polling and only scale to a multi threaded solution if you find that tasks are not getting completed fast enough, and the server had processing power on deck to use.

    Hope this helps! Smile

    Windows Service is perfect for this task but you can't use Windows Service on shared web host providers.

    Alexei Cioina

    Easy Web Reports


    Wednesday, May 19, 2010 9:49 AM
  • User-952121411 posted

    Windows Service is perfect for this task but you can't use Windows Service on shared web host providers.
     

    True.  Read this for more: http://forums.asp.net/t/1558919.aspx#3848444  However, I read all of the OP's posts and I didn't see anywhere it mentioned that this project was being hosted by a 3rd party.  In fact if the following is true:

    I have more then one million users...

    then this is probably a massive site that has their own dedicated servers I would hope.

     

    Wednesday, May 19, 2010 1:39 PM
  • User84821097 posted

    This is very typical problem we face, to handle long running processes over net. There are many work around for it, but i feel when it is not possible for us to reduce the process timeout that also without Session Timeout, browser time out. Even scriptmanager will also fail for the same. In such case we have to find out some other work around which actually sounds good for me, check out the following link.

    http://aspdotnetmatters.blogspot.com/2010/08/aspnet-handling-long-running-process-in.html

    I personally implemented this solution for one of long running process and it is fantastic solution. check it out.

    Cool

    Friday, September 17, 2010 8:48 AM
  • User1137074964 posted

    >I personally implemented this solution for one of long running process and it is fantastic solution.

    Hi, I'm very interested to know if a long running proces take 10 hours then can somebody make requests to your website. If so, then how fast  is your website response.

    Thank you very much!

    Friday, September 17, 2010 3:29 PM
  • User-952121411 posted

    The process that @ashishsarda99 reccomends in the blog post is to use Asynchronous Delegates to wait for the callback.  This is a decent technique for a single long running process in a web app that you do not want to wait for in page or server response.  I use them myself in things like sending an email where I don't want the page's response to be held up by the SMTP server, so I call the method to send the email using an asynchronous delegate. This solution is built into .asmx and WCF service calls too as you will see your methods ending in 'Async' for the Asynchronous version of the call to the service as well.

    However, for processing multiple long running or application wide long running tasks, I think these sorts of things should still be offloaded to a service (i.e. a Windows Service or similar solution) as the recomended link from @ashishsarda99 must run inside the bounds of session timeout (even if the timer elapse resets timeout value), user's care to keep the site open, etc.  So if the task takes 50 minutes as the OP stated, relying on the user to keep the browser open while a session variable and timer keep checking to see if the callback method has fired is not reliable.

    Saturday, September 18, 2010 5:29 PM
  • User84821097 posted

    @cioina To Handle 10 Hrs process will require some different kind of logic, as we can keep session alive, which is not the issue but I feel in that case the work around should be different one. The solution which I have provided is for the single long running process per session (actually did not try out for multiples). But as @atconway said, we can always go for windows services. Also we can think of going for entirely different point of view, i.e. if maintaining session is mandatory we can change the way for storing session.

    Also keeping browser open for 50mins is big deal, if we are thinking of 10 hrs system then, it either should be a real time operation where keeping browser open for such a long time is mandatory.

    Sunday, September 19, 2010 4:32 AM
  • User-1429898154 posted

    you can use Asynchronous page. Check the below link, it is a step by step tutorial showing how you can use it for a long running processes.


    http://intellecting.net/blog/post/High-Performance-ASPNet-Applications-Part-2.aspx

    Sunday, September 19, 2010 7:08 AM
  • User1137074964 posted

    @cioina To Handle 10 Hrs process will require some different kind of logic, as we can keep session alive, which is not the issue but I feel in that case the work around should be different one. The solution which I have provided is for the single long running process per session (actually did not try out for multiples). But as @atconway said, we can always go for windows services. Also we can think of going for entirely different point of view, i.e. if maintaining session is mandatory we can change the way for storing session.

    Also keeping browser open for 50mins is big deal, if we are thinking of 10 hrs system then, it either should be a real time operation where keeping browser open for such a long time is mandatory.

    >should be a real time operation

    My website is on shated hosting provider. No Windows services are allowed. I want to send 100,000 emails which can take more than 24 hours. I want my website to works nolmaly during this process.

    Sunday, September 19, 2010 9:24 AM
  • User-319574463 posted

    My website is on shated hosting provider. No Windows services are allowed. I want to send 100,000 emails which can take more than 24 hours. I want my website to works nolmaly during this process.

    You could write a web service that sends out about 5 seconds worth of email and call it every 10 seconds from a Windows Service on your home PC. That would allow your web site to function more or less normally.

    Tuesday, September 21, 2010 7:21 AM
  • User-481631678 posted

    You can create a thread in Application_Start event in global.asax like this:-

     protected void Application_Start(object sender, EventArgs e)
            {
                Thread mailThread; mailThread = new System.Threading.Thread(delegate()
                {
                    ProcessBackgroundActivities();
                });
                mailThread.IsBackground = true; mailThread.Start();
            }


    and Here is a sample body for ProcessBackgroundActivities:-

    private static void ProcessBackgroundActivities()
    {
    while(true)
    {
    //Check If Any record here to process
    //If yes then run that on a seperate thread
    
    Thread.Sleep(10000);
    }
    }

    Only disadvantage with this approach is that if IIS is stopped then all running thread will be terminated. Also if site is IDLE for some time then IIS might stop your application and BackgroundActivities thread will also be stopped.

    Wednesday, September 22, 2010 1:15 AM
  • User-952121411 posted

    My website is on shated hosting provider. No Windows services are allowed. I want to send 100,000 emails which can take more than 24 hours. I want my website to works nolmaly during this process.
     

    (2) reccomendations:

    1. I didn't realize it at 1st by you are not the original poster of this thread and we have gone off on a separate issue.  You might consider starting a new thread with your specific scenario that will get more visibility.

    2. Just briefly - If you must have an IIS hosted solution for solving this problem of sending a massive amount of emails, I wouldn't use an ASP.NET solution.  I would use a WCF service hosted in IIS to preform all of the email sending, polling, etc.

     

    Thursday, September 23, 2010 10:36 AM
  • User-1175999874 posted

    You may refer to the following link, the best-ever solution to monitor/cancel long-running tasks on a Server:

    http://mehranhoodeh.com/InId_ProgressBarOnServerProcesses.aspx

    Progress Bar on Long-Running Tasks on Servers in Asp.net

    The sample shown above is a simple long-running task to generate a report, but it can be used on any type of long-running tasks; for example, a batch-update on the whole records of a table (imagine millions of records), a bunch of Emails to send, a complex computational algorithm, or any other similar time-consuming ones. I have implemented a class for the ProgressBar, named InnovativeProgressBar, which handles showing the progression of a task in a web page, coded in an straight-forward manner as shown below:

    protected void btnGo_Click(object sender, EventArgs e)
    {
            InnovativeProgressBar ipb = new InnovativeProgressBar("Inventory Exploring");
            ipb.Initialize();
            ipb.SetTotal(records_count);
    
            for (int i = 0; i < records_count; i++)
            {
                ipb.SetCurrent(i, "Processing Rows...");
                
                //  .
                //  .
                //  .
                // [ your Task codes ]
                //  .
                //  .
                //  .
            }//for
    
            ipb.SetComplete();
    }
    

    If you Google the phrase "InnovativeProgressBar" (with double-quotations), you will reach to the page addressed above (in my web-site).

    You may contact me on my web-site, for more information on how to implement it.

    Monday, December 14, 2015 8:51 AM