none
c# Collection was modified. enumeration operation might not execute. RRS feed

  • Question

  • Below is sample code which throwing error called Collection was modified. enumeration operation might not execute.

    am i getting this error because i use AsParallel ?

    .AsParallel().WithDegreeOfParallelism(5)

    string StandardValue = ""
    
    StandardValue = dtFilterDataFromAllData.AsEnumerable().AsParallel().WithDegreeOfParallelism(noofththreads)
    	.Where(w => w.Field<string>("BRTab") == BRTab
    	&& w.Field<string>("RowNumber") == RowNumber
    	&& w.Field<string>("StandardDate").Replace("A", "").Replace("E", "") == strPeriod.Replace("A", "").Replace("E", ""))
    	.Select(v => v.Field<string>("StandardValue"))
    	.FirstOrDefault();
    
    
    if (StandardValue != null && StandardValue.ToString() != "")
                    {
                        //update Standard Value after processing
                        StandardValue = ProcessValue(StandardValue ?? "", strUnits, false, TabName, StandardLineItem, strPeriod.Replace("A", string.Empty).Replace("E", string.Empty));
    
                        dtFilterDataFromAllData.AsEnumerable().Where(w => w.Field<string>("BRTab") == BRTab
                            && w.Field<string>("RowNumber") == RowNumber
                            && w.Field<string>("StandardDate").Replace("A", "").Replace("E", "") == strPeriod.Replace("A", "").Replace("E", ""))
                            .ForEach(w =>
                            {
                                //update parent StandardValue
                                w["StandardValue"] = StandardValue;
                            });
                        dtFilterDataFromAllData.AcceptChanges();
                    }
    can't i use .AsParallel in my above code ? thanks

    Friday, January 31, 2020 12:51 PM

Answers

  • The default implementation of IEnumerable used in most of the types defined in the framework versions the collection being enumerated. If any changes occur after the enumeration is started then the next time a value is requested it throws this exception. This is by design because modifying a collection while enumerating it can cause elements to be skipped or enumerated more than once.

    As Abigail mentions, you need to separate enumeration from modification. There are 2 different common approaches.

    1. Force a full enumeration of the items into a temporary list/array first. Then enumerate the temp list and modify the original collection. This works well for small collections and when you can easily map objects that in a temp list to the original list.

    2. Create a "modify" list that starts out empty. Enumerate the original collection and determine which items need to be modified. Put that item on the modify list. If you need to support both addition and removal then you'll either need separate lists or some sort of way to tell the difference. After you've enumerated the original collection then enumerate the modify list and apply the modifications to the original collection. This works well if there are a lot of items but requires more code. It also doesn't work well if there are dependencies (e.g. remove A if B is also removed).

    Be careful of deferred execution in LINQ though. Any method returning IEnumerable<T> will likely defer the execution until later. Therefore you need to ensure that you don't modify any of the items until after the execution has occurred. In general "To" methods (e.g. ToList, ToArray) and methods that do not return IEnumerable<T> will trigger execution.

    Looking at your posted code the AsParallel block of code is fine because you're simply selecting a value. When the call returns you'll have a single result. However if you're using a data table then parallel selects probably aren't worth anything. Unless you have 1000s of rows in your table (which is an utter misuse of a datatable and an immeasurable waste of resources) then you aren't going to gain anything by doing this complex code over a simple select. Furthermore your FirstOrDefault is going to be inconsistent because the results will be returned in arbitrary order. Unless you can guarantee there is a single row that matches your request you may get different results each time you run. If there is a single row then you are potentially enumerating more rows (and therefore wasting time) trying to find that one row in parallel then you would sequentially. Imagine for example that you have 100 rows and  it is partitioned into 10 batches. Before you can select the first row it has to enumerate all of them because it will enumerate each batch in parallel and then return back any matches. Those matches are then put together (10 rows) so you can select the first one. This is a huge waste of time. If the row you want is the first row then you just wasted 99 enumerations to find it. Compare that to getting rid of the parallel and just searching until you find the first row. Statistically speaking you can expect to search half the rows before you find it so instead of 100 rows you'd search 50. I really, really think you need to rethink the use of parallel here, irrelevant of everything else going on.

    The second block seems OK but I'd have to see it run. It isn't modifying any data other than a field which is fine. It isn't until you call AcceptChanges that any modifications occur to the DataTable itself but at that point there is no more enumeration.


    Michael Taylor http://www.michaeltaylorp3.net

    • Marked as answer by Sudip_inn Friday, January 31, 2020 8:49 PM
    Friday, January 31, 2020 3:11 PM
    Moderator
  • Use a for-next instead of a ForEach as the ForEach can not be updated as per you altering the data.

    Please remember to mark the replies as answers if they help and unmarked them if they provide no help, this will help others who are looking for solutions to the same or similar problem. Contact via my Twitter (Karen Payne) or Facebook (Karen Payne) via my MSDN profile but will not answer coding question on either.

    NuGet BaseConnectionLibrary for database connections.

    StackOverFlow
    profile for Karen Payne on Stack Exchange

    • Marked as answer by Sudip_inn Friday, January 31, 2020 8:50 PM
    Friday, January 31, 2020 4:12 PM
    Moderator
  • if i use for instead of foreach then my code will work fine ?
    It should, only way to know is to refactor it as a for rather than a for-each and run the code.

    Please remember to mark the replies as answers if they help and unmarked them if they provide no help, this will help others who are looking for solutions to the same or similar problem. Contact via my Twitter (Karen Payne) or Facebook (Karen Payne) via my MSDN profile but will not answer coding question on either.

    NuGet BaseConnectionLibrary for database connections.

    StackOverFlow
    profile for Karen Payne on Stack Exchange

    • Marked as answer by Sudip_inn Saturday, February 1, 2020 7:05 PM
    Friday, January 31, 2020 8:54 PM
    Moderator
  • List<SomeData> data = ...;
    
    //Create a temp list
    var items = data.ToArray();
    
    //Enumerate the temp list 
    foreach (var item in items)
    {
       //Remove from original list on some condition
       if (SomeCondition(item))
          data.Remove(item);
    };
    
    





    Michael Taylor http://www.michaeltaylorp3.net

    • Marked as answer by Sudip_inn Saturday, February 1, 2020 7:06 PM
    Friday, January 31, 2020 9:46 PM
    Moderator

All replies

  • Normally, if I am going to change an item in collection using a for loop, it will be a for loop using an index. As I recall,  using a foreach loop doen't allow for changing of an item in the collection.
    Friday, January 31, 2020 2:09 PM
  • When you try to replace rows when enumeration in a for each loop, you will get that error.
    To replace rows from datatable, you could use the logic mentioned below -
    Use the select function on the datatable to filter the rows that match your condition and this will return an array of Datarows. Then use a for each loop and remove each row from the Array of Datarows using datatable.Rows.Remove(row);

    Hope this helps.

    Friday, January 31, 2020 2:30 PM
  • The default implementation of IEnumerable used in most of the types defined in the framework versions the collection being enumerated. If any changes occur after the enumeration is started then the next time a value is requested it throws this exception. This is by design because modifying a collection while enumerating it can cause elements to be skipped or enumerated more than once.

    As Abigail mentions, you need to separate enumeration from modification. There are 2 different common approaches.

    1. Force a full enumeration of the items into a temporary list/array first. Then enumerate the temp list and modify the original collection. This works well for small collections and when you can easily map objects that in a temp list to the original list.

    2. Create a "modify" list that starts out empty. Enumerate the original collection and determine which items need to be modified. Put that item on the modify list. If you need to support both addition and removal then you'll either need separate lists or some sort of way to tell the difference. After you've enumerated the original collection then enumerate the modify list and apply the modifications to the original collection. This works well if there are a lot of items but requires more code. It also doesn't work well if there are dependencies (e.g. remove A if B is also removed).

    Be careful of deferred execution in LINQ though. Any method returning IEnumerable<T> will likely defer the execution until later. Therefore you need to ensure that you don't modify any of the items until after the execution has occurred. In general "To" methods (e.g. ToList, ToArray) and methods that do not return IEnumerable<T> will trigger execution.

    Looking at your posted code the AsParallel block of code is fine because you're simply selecting a value. When the call returns you'll have a single result. However if you're using a data table then parallel selects probably aren't worth anything. Unless you have 1000s of rows in your table (which is an utter misuse of a datatable and an immeasurable waste of resources) then you aren't going to gain anything by doing this complex code over a simple select. Furthermore your FirstOrDefault is going to be inconsistent because the results will be returned in arbitrary order. Unless you can guarantee there is a single row that matches your request you may get different results each time you run. If there is a single row then you are potentially enumerating more rows (and therefore wasting time) trying to find that one row in parallel then you would sequentially. Imagine for example that you have 100 rows and  it is partitioned into 10 batches. Before you can select the first row it has to enumerate all of them because it will enumerate each batch in parallel and then return back any matches. Those matches are then put together (10 rows) so you can select the first one. This is a huge waste of time. If the row you want is the first row then you just wasted 99 enumerations to find it. Compare that to getting rid of the parallel and just searching until you find the first row. Statistically speaking you can expect to search half the rows before you find it so instead of 100 rows you'd search 50. I really, really think you need to rethink the use of parallel here, irrelevant of everything else going on.

    The second block seems OK but I'd have to see it run. It isn't modifying any data other than a field which is fine. It isn't until you call AcceptChanges that any modifications occur to the DataTable itself but at that point there is no more enumeration.


    Michael Taylor http://www.michaeltaylorp3.net

    • Marked as answer by Sudip_inn Friday, January 31, 2020 8:49 PM
    Friday, January 31, 2020 3:11 PM
    Moderator
  • suppose this way i am updating my datatable

    dtFilterDataFromAllData.AsEnumerable()
    .Where(a => a.Field<string>("TabName") == TabName
    && a.Field<string>("StandardLineItem") == StandardLineItem
    && a.Field<bool>("cumulative").ToString().ToUpper() == "TRUE"
    && a.Field<string>("StandardDate") == StandardDate.Replace("1Q", "2Q")).ToList<DataRow>()
    .ForEach(y =>
    {
    	y["StandardValue"] = dblcumulativeValue;
    });
    dtFilterDataFromAllData.AcceptChanges();

    in above code if i use AsParallel then getting exception. may be encounter this exception 

    Collection was modified. enumeration operation might not execute

    so how to use PLINQ during updation of datatable.

    please help me with sample code using PLINQ for update. thanks


    Friday, January 31, 2020 3:21 PM
  • Use a for-next instead of a ForEach as the ForEach can not be updated as per you altering the data.

    Please remember to mark the replies as answers if they help and unmarked them if they provide no help, this will help others who are looking for solutions to the same or similar problem. Contact via my Twitter (Karen Payne) or Facebook (Karen Payne) via my MSDN profile but will not answer coding question on either.

    NuGet BaseConnectionLibrary for database connections.

    StackOverFlow
    profile for Karen Payne on Stack Exchange

    • Marked as answer by Sudip_inn Friday, January 31, 2020 8:50 PM
    Friday, January 31, 2020 4:12 PM
    Moderator
  • Why do you keep posting the same question you have already asked in another post?
    Friday, January 31, 2020 5:56 PM
  • please provide sample code for your point 1

    1. Force a full enumeration of the items into a temporary list/array first. Then enumerate the temp list and modify the original collection. This works well for small collections and when you can easily map objects that in a temp list to the original list.

    Friday, January 31, 2020 8:49 PM
  • if i use for instead of foreach then my code will work fine ?
    Friday, January 31, 2020 8:51 PM
  • if i use for instead of foreach then my code will work fine ?
    It should, only way to know is to refactor it as a for rather than a for-each and run the code.

    Please remember to mark the replies as answers if they help and unmarked them if they provide no help, this will help others who are looking for solutions to the same or similar problem. Contact via my Twitter (Karen Payne) or Facebook (Karen Payne) via my MSDN profile but will not answer coding question on either.

    NuGet BaseConnectionLibrary for database connections.

    StackOverFlow
    profile for Karen Payne on Stack Exchange

    • Marked as answer by Sudip_inn Saturday, February 1, 2020 7:05 PM
    Friday, January 31, 2020 8:54 PM
    Moderator
  • List<SomeData> data = ...;
    
    //Create a temp list
    var items = data.ToArray();
    
    //Enumerate the temp list 
    foreach (var item in items)
    {
       //Remove from original list on some condition
       if (SomeCondition(item))
          data.Remove(item);
    };
    
    





    Michael Taylor http://www.michaeltaylorp3.net

    • Marked as answer by Sudip_inn Saturday, February 1, 2020 7:06 PM
    Friday, January 31, 2020 9:46 PM
    Moderator