none
displaying a progress bar during data transfer RRS feed

  • Question

  • Hello

    In this snippet below this post, the data transfer occurs on the last line and take about 60 seconds.
    Would it be possible to display a progress bar on a for to show the progress of the data transfer?
    In other words, is there some infrastrure functionality allowing to observe the progress from another thread?

    Thanks,

    Michel

    ______________________________________________________________________________
    Code snippet

    DataClasses1    DataContext   dt = new DataClasses1DataContext();
    var tbl1  =  dt.myDataTable;
    var tbl2  =  tbl1.ToArray();            // <= data transfer occurs here for about 60 seconds

    • Edited by Lalbatros Friday, June 11, 2010 8:31 PM font size
    Friday, June 11, 2010 8:29 PM

Answers

  • You're loading the entire table - based on the time it is taking I am assuming it is a fairly large amount of data.

    Do you need the entire table in-memory? Are you really using all data in the table in your app? If so, you may want to use .Skip+.Take paging to get chunks of data, and can update a progress bar for every chunk... E.g:

    List<MyDataTable> someData = new List<MyDataTable>();
    int rowCount = dt.myDataTable.Count();

    //TODO: <= display a progress bar here, and set max to rowCount...

    int currentRows = 0;
    while (currentRows < rowCount)
    {
      someData.AddRange(dt.myDataTable.Skip(currentRows).Take(10000).ToList());
      currentRows = someData.Count;

      //TODO: <= update progress here...

    }


    Kristofer - Huagati Systems Co., Ltd.
    Cool tools for Linq-to-SQL and Entity Framework:
    huagati.com/dbmltools (add-in with new features for the L2S and EF designers in VS2008 and VS2010)
    huagati.com/L2SProfiler (Query profiler for Linq-to-SQL and LLBLGen Pro)
    Monday, June 14, 2010 7:04 AM
    Answerer
  • However, I would be curious to learn if and how data object could be shared between different threads,
    and if one thred could "see" the a DataTable being progressively populated in another thread.

     


    If instead of calling .ToArray or .ToList you iterate through the enumerable and add items to a List (with the usual thread sync necessities added) you could monitor the progress from another thread.

    Still, do you really need the entire table in memory? If you're going to process a very large amount of records you might as well process the data as you're enumerating it...


    Kristofer - Huagati Systems Co., Ltd.
    Cool tools for Linq-to-SQL and Entity Framework:
    huagati.com/dbmltools (add-in with new features for the L2S and EF designers in VS2008 and VS2010)
    huagati.com/L2SProfiler (Query profiler for Linq-to-SQL and LLBLGen Pro)
    Tuesday, June 22, 2010 9:53 AM
    Answerer

All replies

  • Hi,

    As far as I know there's no such an infrastructure so that you can query a percentage for the data which will be returned after executing a SQL query.

    To inform the user from the task which is going to take time in background, you can use a ProgressBar control and having it's Style property set to Marquee.

     

    Hope it helps


    learn to learn
    Sunday, June 13, 2010 11:06 AM
  • You're loading the entire table - based on the time it is taking I am assuming it is a fairly large amount of data.

    Do you need the entire table in-memory? Are you really using all data in the table in your app? If so, you may want to use .Skip+.Take paging to get chunks of data, and can update a progress bar for every chunk... E.g:

    List<MyDataTable> someData = new List<MyDataTable>();
    int rowCount = dt.myDataTable.Count();

    //TODO: <= display a progress bar here, and set max to rowCount...

    int currentRows = 0;
    while (currentRows < rowCount)
    {
      someData.AddRange(dt.myDataTable.Skip(currentRows).Take(10000).ToList());
      currentRows = someData.Count;

      //TODO: <= update progress here...

    }


    Kristofer - Huagati Systems Co., Ltd.
    Cool tools for Linq-to-SQL and Entity Framework:
    huagati.com/dbmltools (add-in with new features for the L2S and EF designers in VS2008 and VS2010)
    huagati.com/L2SProfiler (Query profiler for Linq-to-SQL and LLBLGen Pro)
    Monday, June 14, 2010 7:04 AM
    Answerer
  • Hi Michel,

     

    I am writing to check the status of the issue on your side.  Would you mind letting us know the result of the suggestions? 

     

    If you need further assistance, please feel free to let me know.   I will be more than happy to be of assistance.

     

    Have a nice day!

     

     

    Best Regards,
    Lingzhi Sun

    MSDN Subscriber Support in Forum

    If you have any feedback on our support, please contact msdnmg@microsoft.com


    Please remember to mark the replies as answers if they help and unmark them if they provide no help.
    Welcome to the All-In-One Code Framework! If you have any feedback, please tell us.
    Monday, June 21, 2010 9:10 AM
    Moderator
  • Hi Lingzhi,

    These answers were sufficient for the time being.

    However, I would be curious to learn if and how data object could be shared between different threads,
    and if one thred could "see" the a DataTable being progressively populated in another thread.

    Thanks,

    Michel

    Tuesday, June 22, 2010 9:48 AM
  • However, I would be curious to learn if and how data object could be shared between different threads,
    and if one thred could "see" the a DataTable being progressively populated in another thread.

     


    If instead of calling .ToArray or .ToList you iterate through the enumerable and add items to a List (with the usual thread sync necessities added) you could monitor the progress from another thread.

    Still, do you really need the entire table in memory? If you're going to process a very large amount of records you might as well process the data as you're enumerating it...


    Kristofer - Huagati Systems Co., Ltd.
    Cool tools for Linq-to-SQL and Entity Framework:
    huagati.com/dbmltools (add-in with new features for the L2S and EF designers in VS2008 and VS2010)
    huagati.com/L2SProfiler (Query profiler for Linq-to-SQL and LLBLGen Pro)
    Tuesday, June 22, 2010 9:53 AM
    Answerer
  • KristopherA,

    I voted helpful for your post, but to my surprise a -1 vote is being displayed.
    This must be some bug!

    Thanks for your comment,

    Michel

    Tuesday, June 22, 2010 8:32 PM
  • KristopherA,

    I voted helpful for your post, but to my surprise a -1 vote is being displayed.
    This must be some bug!

    Thanks for your comment,

    Michel


    Haha, yes, looks like they have reversed the sign on the votes. :) Maybe they got inspired by upvotes/downvotes at stackoverflow.com...
    Kristofer - Huagati Systems Co., Ltd.
    Cool tools for Linq-to-SQL and Entity Framework:
    huagati.com/dbmltools (add-in with new features for the L2S and EF designers in VS2008 and VS2010)
    huagati.com/L2SProfiler (Query profiler for Linq-to-SQL and LLBLGen Pro)
    Wednesday, June 23, 2010 2:46 AM
    Answerer
  • :-)  It's really strange!   I vote it again and it becomes zero again! 

     

     

    Good day!

     

     

    Best Regards,
    Lingzhi Sun

    MSDN Subscriber Support in Forum                                                                                         

    If you have any feedback on our support, please contact msdnmg@microsoft.com


    Please remember to mark the replies as answers if they help and unmark them if they provide no help.
    Welcome to the All-In-One Code Framework! If you have any feedback, please tell us.
    Wednesday, June 23, 2010 6:41 AM
    Moderator