none
UI freezes while adding large number of records inspite of using backgroundworker / threading

    Question

  •  

    Hello,

     

    I am facing an issue with updating a large number of records to a listview,the aim being not freezing the UI.

     

    I tried the following two solutions but in vain -

     

    1. Using Backgroundworker component, in the dowork method, i am looping thru a datareader and constructing a

        listviewitem. Then i am adding each listview item to a generic list i.e List of (Listviewitems).

        There are close to 10 thousand records. Finally in the report progress method, i am looping through the list of  listviewitems and populating the listview at one shot. It works fine, without any flickering.

        But the problem is  - if i click on the Quit button it does not shut down immediately  - the UI remains frozen until all   the items are added to the listview. I have called backgroundworker.cancelasync in the quit button click, but it does not get called until all items are added to the listview.

     

    2. Another solution was using threading. I spawned another thread and called control.invoke to update the UI

       Here the issue is that the flickering is way too much, but the form quits immediately on clicking quit.

      
       Can someone suggest how I can achieve both addition of items to the UI without flickering and canceling  immediately.

     

    Should i try some other solution?

     

    Please advice.

     

    Thanks,

    Rachel

    Wednesday, August 08, 2007 5:03 PM

Answers

  • I tried out generally what you are doing and I was able to reproduce your freeze-up of the form and control.  I found that if I put a slight delay by having the thread sleep even a tiny amount, it was time enough for the listview control to catch up.  My approach is slightly differnt than yours -- to avoid having a collection that needs to be iterated through (and thus altered while being refrerenced), I create a new listview item from the results of the DoWork operation and pass that along in the UserState object.  That way only one item at a time can be added to the control.

     

    This approach worked for me, and I hope it offers some help for your situation.

     

     

    Code Snippet

    Private Sub btnRun_Click(ByVal sender As System.Object, _

    ByVal e As System.EventArgs) Handles btnRun.Click

      ListView1.Clear()

      bw.RunWorkerAsync()

    End Sub

     

    Private Sub btnCancel_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) _

    Handles btnCancel.Click

      If bw.IsBusy Then bw.CancelAsync()

    End Sub

     

    Private Sub bw_DoWork(ByVal sender As Object, ByVal e As System.ComponentModel.DoWorkEventArgs) _

    Handles bw.DoWork

      For x As Int32 = 1 To 10000

    Dim lvi As New ListViewItem("Item " & x)

    'I need to have some pause here otherwise the control/form locks up

    Threading.Thread.Sleep(10)

    If bw.CancellationPending Then

      e.Cancel = True

      Exit For

    Else

      bw.ReportProgress(0, lvi)

         End If

      Next

    End Sub

     

    Private Sub bw_ProgressChanged(ByVal sender As Object, _

    ByVal e As System.ComponentModel.ProgressChangedEventArgs) _

    Handles bw.ProgressChanged

      Try

    Dim lvi As ListViewItem = DirectCast(e.UserState, ListViewItem)

    Me.ListView1.Items.Add(lvi)

      Catch ex As Exception

    Throw New Exception(ex.Message)

      End Try

    End Sub

     

    Private Sub bw_RunWorkerCompleted(ByVal sender As Object, _

    ByVal e As System.ComponentModel.RunWorkerCompletedEventArgs) _

    Handles bw.RunWorkerCompleted

      If Not e.Cancelled Then

    MessageBox.Show("Done")

      Else

    MessageBox.Show("Cancelled")

      End If

    End Sub

     

     

    Thursday, August 09, 2007 12:19 PM

All replies

  •  

    Use Application.DoEvents after each record.

     

    Wednesday, August 08, 2007 6:21 PM
  • I'm surprised the CancelAsync option didn't work.  It sounds as though you should have plenty of opportunity to check the background worker to see if it has been a pending cancel -- you could do it within each loop of the datareader.  Are you checking the BackgroundWorker.CancellationPending property at a frequent rate?  It's not enough to call the CancelAsync method -- you ned to have the worker check it periodically.  When it sees that there is a pending request, then you can have it kick out of the loop and stop performing any more operations.  In the RunWorkerCompleted event of the worker, you can do clean up before allowing the app to close (if the e.IsCancelled property in the event args indicates it was cancelled).

    Wednesday, August 08, 2007 6:25 PM
  • I already tried that. While using application.doevents - it pops up an error saying - "Collection was modified. Enumeration operation may not execute"

    Any idea what the reason could be?

    Thursday, August 09, 2007 3:28 AM
  • The reason CancelAsync is not working is because, the UI is frozen and until the population of data is over, it does not register the click.

     

    This is the code that i have written -

     

    Private Sub bgWorker_ProgressChanged(ByVal sender As System.Object, ByVal e As System.ComponentModel.ProgressChangedEventArgs) Handles bgWorker.ProgressChanged

    Try

    Dim lvi As ListViewItem

    For Each lvi In DirectCast(e.UserState, List(Of ListViewItem))

    Application.DoEvents()

    If bgWorker.CancellationPending Then

    Exit Sub

    End If

    lvi = DirectCast(lvi, ListViewItem)

    Me.lvwVSUMM_LIST.Items.Add(lvi)

    Next

    Catch ex As Exception

    throw new Exception(ex.Message)

    End Try

     

    This is the code for Quit  button -

     

    If bgWorker.IsBusy Then

    bgWorker.CancelAsync()

    End If

     

    Please tell me some way out !

     

    Thanks ..

    Thursday, August 09, 2007 3:33 AM
  • I tried out generally what you are doing and I was able to reproduce your freeze-up of the form and control.  I found that if I put a slight delay by having the thread sleep even a tiny amount, it was time enough for the listview control to catch up.  My approach is slightly differnt than yours -- to avoid having a collection that needs to be iterated through (and thus altered while being refrerenced), I create a new listview item from the results of the DoWork operation and pass that along in the UserState object.  That way only one item at a time can be added to the control.

     

    This approach worked for me, and I hope it offers some help for your situation.

     

     

    Code Snippet

    Private Sub btnRun_Click(ByVal sender As System.Object, _

    ByVal e As System.EventArgs) Handles btnRun.Click

      ListView1.Clear()

      bw.RunWorkerAsync()

    End Sub

     

    Private Sub btnCancel_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) _

    Handles btnCancel.Click

      If bw.IsBusy Then bw.CancelAsync()

    End Sub

     

    Private Sub bw_DoWork(ByVal sender As Object, ByVal e As System.ComponentModel.DoWorkEventArgs) _

    Handles bw.DoWork

      For x As Int32 = 1 To 10000

    Dim lvi As New ListViewItem("Item " & x)

    'I need to have some pause here otherwise the control/form locks up

    Threading.Thread.Sleep(10)

    If bw.CancellationPending Then

      e.Cancel = True

      Exit For

    Else

      bw.ReportProgress(0, lvi)

         End If

      Next

    End Sub

     

    Private Sub bw_ProgressChanged(ByVal sender As Object, _

    ByVal e As System.ComponentModel.ProgressChangedEventArgs) _

    Handles bw.ProgressChanged

      Try

    Dim lvi As ListViewItem = DirectCast(e.UserState, ListViewItem)

    Me.ListView1.Items.Add(lvi)

      Catch ex As Exception

    Throw New Exception(ex.Message)

      End Try

    End Sub

     

    Private Sub bw_RunWorkerCompleted(ByVal sender As Object, _

    ByVal e As System.ComponentModel.RunWorkerCompletedEventArgs) _

    Handles bw.RunWorkerCompleted

      If Not e.Cancelled Then

    MessageBox.Show("Done")

      Else

    MessageBox.Show("Cancelled")

      End If

    End Sub

     

     

    Thursday, August 09, 2007 12:19 PM
  • Thanks a bunch for the response. It solves the problem of the UI freezing, but since each record is loaded one by one, the time taken is painfully long. It takes about a second to load a record, so you can imagine the time it is taking to load 10000 records. That is the reason why i added the records to a list and then added it to the listview at a go.

     

    Can u think of some other alternative ?

     

     

    Tuesday, August 14, 2007 5:34 AM
  • With that kind of load time and so many records to load have you considered changing your project design?  Or at the very least consider alternate, much faster ways to read in your data.  Do you absolutely have to show all 10,000 records at once?  As a serious design question, who wants to scan through 10,000 records anyhow?  Can you ask the user to offer filters to what data is queried (i.e. by date range or some other filterable value)?

    For instance, say these were poeple's names being loaded. You could have a series of buttons (or some other method of choosing) that correspond to the first letter of the name, or cluster the letters in logical partitions.

    Tuesday, August 14, 2007 7:31 PM
  • Instead of using a data reader and accesing the records one by one...have you tried using filling the data in a data set and then loading the ListView with that data ...but u have to use List View in virtual mode to get a faster response.If u use ListView in non-Virtual mode then it will take very long time compared to Vitual mode...
    Sunday, August 19, 2007 5:10 PM
  • I am experiencing same problem, my application has a continuous For loop that stops when condition is met.
    But when it starts executing, UI freezes, the Thread.Sleep method doent worked.
    My application needs to access one record at a time. So i am using Application.DoEvents() method to Unfreeze UI.
    Whenever the looping statement occur within threads the UI freezes (my observation).

    So how can we replace the looping statements?

    Sunday, September 02, 2007 1:18 PM
  • The reason the UI thread freezes is because it is doing nothing but processing the requests to execute the ProgressChanged event.  If you call ReportProgress more than a couple of hundred times per second, the UI thread will essentially die, not getting around handling Click and Paint events.  You'll have to slow down the rate at which you call ProgressChanged.  The Sleep() call does this but at an obvious and severe loss of performance on the background thread.

    The solution is to let your BGW collect the results of the query.  A collection class like List(Of MyRecord) is a good type for that.  It should then not call ReportProgress until enough time has expired.  Here's an example:

      Private Sub BackgroundWorker1_DoWork(ByVal sender As System.Object, ByVal e As System.ComponentModel.DoWorkEventArgs) Handles BackgroundWorker1.DoWork
        Dim lastProgress As DateTime = Now
        Dim results As New List(Of MyRecord)
        Dim cmd As New SqlClient.SqlCommand
        '--- Open your query here...
        '...
        Using rs As SqlClient.SqlDataReader = cmd.ExecuteReader()
          Do While rs.Read()
            If BackgroundWorker1.CancellationPending Then
              '--- Quit if user requested cancel, cleanup dbase resources if necessary
              e.Cancel = True
              Return
            End If
            Dim rec As New MyRecord
            '--- Copy query result into <rec> and add to results
            '...
            results.Add(rec)
            '--- Let UI thread use results if enough time expired
            If (Now - lastProgress).TotalMilliseconds >= 42 Then
              BackgroundWorker1.ReportProgress(0, results)
              lastProgress = Now
              results.Clear()
            End If
          Loop
          '--- Final callback for last results
          If results.Count > 0 Then BackgroundWorker1.ReportProgress(0, results)
        End Using
      End Sub

      Private Sub BackgroundWorker1_ProgressChanged(ByVal sender As System.Object, ByVal e As System.ComponentModel.ProgressChangedEventArgs) Handles BackgroundWorker1.ProgressChanged
        Dim result As List(Of MyRecord) = CType(e.UserState, List(Of MyRecord))
        '--- Use result to update UI controls
        '...
      End Sub

      Private Sub BackgroundWorker1_RunWorkerCompleted(ByVal sender As System.Object, ByVal e As System.ComponentModel.RunWorkerCompletedEventArgs) Handles BackgroundWorker1.RunWorkerCompleted
        If e.Cancelled Then
          '--- Cancelled, clean up intermediary results, notify user
          '...
        Else
          '--- All's well, enable editing controls etc.
          '...
        End If
      End Sub
    Sunday, September 02, 2007 4:05 PM