none
Generic.List<string> + Remove and Count RRS feed

  • Question

  • Hi,

    I am having code like this:

    System.Collections.Generic.List<string> newFiles = new System.Collections.Generic.List<string>();

    I am populating newFiles list object with filenames from a folder:

    while (fileDate.Date <= currentDay.Date)
                {
                    string filter = rootSourceFileName + "*" + fileDate.ToString("yyyyMMdd") + "*.txt";
                    string[] listOfFiles = Directory.GetFiles(sourcePath, filter);
                    foreach (string s in listOfFiles)
                        newFiles.Add(System.IO.Path.GetFileName(s));

                    fileDate = fileDate.AddDays(1);
                }       

    After this, I need to make sure NewFiles list are already not processed. For this, we are keeping track of file names in the database. So, I get list of processed file names from database into a table. I loop through each row in data table and check against newFiles list and remove it from newFiles. Removing works but count is not changing after removal.

                     foreach (DataRow dr in dt.Rows)
                {
                    newFiles.Remove(sourcePath + dr[0].ToString());
                }

                // Return the ArrayList which will only have new files.
                 Dts.Variables["Count"].Value = newFiles.Count;

    This newFiles.Count is not changing after removal of file from newFiles object. Any idea why count is not getting updated.

    Thank You

    Thursday, April 25, 2019 4:27 PM

Answers

  • If Count isn't changing then nothing is being removed with the call to Remove. The most likely culprit I can think of is casing. `MyData.txt` != `mydata.txt` So nothing is being removed.

    Since you're working with List<T> then you can cheat and use a helper method RemoveAll.

    //Replace the foreach with a simple LINQ to get all the values from the first column of each row
    var oldFiles = dt.Rows.OfType<DataRow>().Select(r => r[0].ToString());
    
    //Remove all entries from newFiles that are contained in oldFiles
    newFiles.RemoveAll(f => oldFiles.Contains(f, StringComparer.OrdinalIgnoreCase));
    Note also that your original loop is pretty inefficient since you're enumerating the directory structure for each file for each day. It would be more efficient to grab all the files from the directory and then enumerate through and get the files you want. In your case it seems like you'll match files of the format (<base>*<date>) so you can filter on the front half of the filename (and extension) and then use a filter to eliminate days you don't care about. Of course if you're only talking about 100 files or so then the performance probably won't be noticeable.


    Michael Taylor http://www.michaeltaylorp3.net

    • Marked as answer by Spunny Thursday, April 25, 2019 8:07 PM
    Thursday, April 25, 2019 5:50 PM
    Moderator
  • NewFiles didn't have exact data as  sourcePath + dr[0].ToString();

    So nothing was getting removed and count was not getting refreshed. Took care of issue and count now reflects change.

    • Marked as answer by Spunny Thursday, April 25, 2019 8:09 PM
    Thursday, April 25, 2019 8:08 PM

All replies

  • If Count isn't changing then nothing is being removed with the call to Remove. The most likely culprit I can think of is casing. `MyData.txt` != `mydata.txt` So nothing is being removed.

    Since you're working with List<T> then you can cheat and use a helper method RemoveAll.

    //Replace the foreach with a simple LINQ to get all the values from the first column of each row
    var oldFiles = dt.Rows.OfType<DataRow>().Select(r => r[0].ToString());
    
    //Remove all entries from newFiles that are contained in oldFiles
    newFiles.RemoveAll(f => oldFiles.Contains(f, StringComparer.OrdinalIgnoreCase));
    Note also that your original loop is pretty inefficient since you're enumerating the directory structure for each file for each day. It would be more efficient to grab all the files from the directory and then enumerate through and get the files you want. In your case it seems like you'll match files of the format (<base>*<date>) so you can filter on the front half of the filename (and extension) and then use a filter to eliminate days you don't care about. Of course if you're only talking about 100 files or so then the performance probably won't be noticeable.


    Michael Taylor http://www.michaeltaylorp3.net

    • Marked as answer by Spunny Thursday, April 25, 2019 8:07 PM
    Thursday, April 25, 2019 5:50 PM
    Moderator
  • thank you Michael. I will try
    Thursday, April 25, 2019 8:07 PM
  • NewFiles didn't have exact data as  sourcePath + dr[0].ToString();

    So nothing was getting removed and count was not getting refreshed. Took care of issue and count now reflects change.

    • Marked as answer by Spunny Thursday, April 25, 2019 8:09 PM
    Thursday, April 25, 2019 8:08 PM