none
Odd intermittent Dictionary Error. Can anyone help? RRS feed

  • Question

  • Hi,

    I get the following exception related to following method on an intermittent basis and I am unsure what could be causing the error.  The line related to the error message is highlighted below. Any insights would be appreciated.

    Paul

    ---- Error Message

    Unhandled Exception:

    System.IndexOutOfRangeException: Index was outside the bounds of the array.
       at System.Collections.Generic.Dictionary`2.Enumerator.MoveNext()
       at DirUtil.DirUtil.DictionaryComp(Dictionary`2 LeftSide, Dictionary`2 RightSide, String AppUser) in Q:\DirUtil\DirUtil\DirUtil.cs:line 988
       at DirUtil.DirUtil.Main() in Q:\DirUtil\DirUtil\DirUtil.cs:line 1700

    --- method

            public static void DictionaryComp(Dictionary<string, string> LeftSide, Dictionary<string, string> RightSide, string AppUser)
            {
                foreach (KeyValuePair<string, string> Items in RightSide)
                {
                    if (Items.Key != null)
                    {
                        if (LeftSide.TryGetValue(Items.Key.ToString(), out string CompValue))
                        {
                            DateTime LeftDate = DateTime.Parse(Items.Value, null, DateTimeStyles.RoundtripKind);
                            DateTime RightDate = DateTime.Parse(CompValue, null, DateTimeStyles.RoundtripKind);
                            int CompResult = DateTime.Compare(LeftDate, RightDate);
                            if (CompResult == 1)
                            {
                                if (!RecordUpdates.ContainsKey(Items.Key))
                                {
                                    RecordUpdates.Add(Items.Key, LeftDate);
                                    UpdateEntity(Items.Key, LeftDate, DirApp, AppUser);
                                }
                            }
                        }
                        else
                        {
                            RecordAdds.Enqueue(Items.Key);
                        }
                    }
                }

            }

    Thursday, May 23, 2019 2:54 PM

Answers

  • Given WalkDictionary, you can get a null key if either SysDir has a key that is null or if it has any items that are null. So I think you need to step back and look at how SysDir is being populated.

    Be aware that you're exposing these things publicly so any other code may reference them as well. Unless they need to be public then make them private.

    Also, you seem to be using a lot of public properties in the code. If you don't need things to be public then don't. This just adds to the frailty of the code. You should start with local variables and then promote to fields if you need the data outside the method (and cannot use a return value). Then you promote a field to a property if you need it outside the containing class.

    Looking at your ProcessDirectory logic I don't understand the catch logic. You start the function by trying to get all the files in it. For each file you replace the drive letter with a file share name and then add it to a list. You're counting the files as well but that seems redundant because you can always get the final count of files at the end (`iFile.Count`).

    If the path is too long then you write to some ELine variable but note that in each case you write a value and then stomp over that value with a new message. The first write to ELine does nothing. After that you call DItems to get a new list of flies and then repeat the same logic you had earlier. So DItems somehow handles these errors? Maybe something simpler, like this (not tested).

    public static void ProcessDirectory ( string targetDirectory )
    {
        var iFile = new List<string>();
                
        // Process the list of files found in the directory
        NumODirs++;
        try
        {                
            var files = TryGetFiles(targetDirectory);
            iFile.AddRange(files);                
        } catch (Exception expt)
        {
            var msg = "Exception in Processing Directory :" + targetDirectory + ". ";
            MessageHandler("ERROR", "Exception Location", msg, "ProcessDirectory", AppUser);
    
            msg = expt.Message.ToString();
            MessageHandler("ERROR", "Exception Detail", msg, "ProcessDirectory", AppUser);
    
            //Note that at this point you have no files so do you really want to continue?
        };
    
        string TDir = UseFileShare(targetDirectory);
    
        if (iFile.Count > 0)
        {
            if (!SysDir.ContainsKey(TDir))
            { SysDir.Add(TDir, iFile); }
        }
    
        // Recurse into subdirectories of this directory            
        try
        {
            var subdirectoryEntries = TryGetDirectories(targetDirectory).ToList();
    
            foreach (var subdirectory in subdirectoryEntries)
                ProcessDirectory(subdirectory);
        } catch (Exception e)
        {
            var msg = "Error in Process Directory :" + targetDirectory.ToString() + " subdirectory processing.";
            MessageHandler("ERROR", "Exception Location", msg, "ProcessDirectory", AppUser);
            MessageHandler("ERROR", "Exception Detail", e.Message, "ProcessDirectory", AppUser);
        }            
    
        NumOFile += iFile.Count;
    }
    
    private static IEnumerable<string> TryGetDirectories ( string targetDirectory )
    {
        try
        {
            return Directory.GetDirectories(targetDirectory);
        } catch (PathTooLongException e)
        {
            return SItems(targetDirectory);
        } catch (DirectoryNotFoundException e)
        {
            return SItems(targetDirectory);
        };
    }
    
    private static IEnumerable<string> TryGetFiles ( string targetDirectory )
    {
        try
        {
            //Using GetFiles which will trigger an immediate enumeration here
            return from f in Directory.GetFiles(targetDirectory)
                    select UseFileShare(f);
        } catch (PathTooLongException)
        {
            return from f in DItems(targetDirectory).ToArray();
        } catch (DirectoryNotFoundException)
        {
            return from f in DItems(targetDirectory).ToArray();                
        };
    }
    
    private static string UseFileShare ( string targetDirectory )
    {
        return targetDirectory.Replace(DriveLetter, FullShare.ToString());
    }


    Michael Taylor http://www.michaeltaylorp3.net

    • Marked as answer by PaulFlores Thursday, September 19, 2019 1:37 PM
    Tuesday, June 4, 2019 2:00 PM
    Moderator
  • The culprit turned out to be some interesting edge cases that testing turned up.  The most problematic edge case would seem to be most obvious dealing with the root of the hierarchy that the folder / directory structure represents.

    Thanks to Mike for his suggestion they were helpful in tracking down the edge cases.

    Marking this answered!

    • Marked as answer by PaulFlores Wednesday, June 12, 2019 9:56 PM
    Wednesday, June 12, 2019 9:56 PM

All replies

  • Is it POSSIBLE that the RightSide dictionary is being modified either by some other thread, or by the code in UpdateEntity?  That's a common way of messing up a loop like this.  If you need to remove entries from the dictionary, you need to create a list of the names to remove, then after the loop is finished, run through the "remove" list.

    Tim Roberts | Driver MVP Emeritus | Providenza &amp; Boekelheide, Inc.

    Thursday, May 23, 2019 6:04 PM
  • Hi PaulFlores,

    Thank for posting here.

    For your question, could you provide the complete code with me including the following keywords and your two dictionaries and one string?

    RecordUpdates, UpdateEntity ,DirApp,RecordAdds

    We will solve your problem in time if you provide the above information.

    Best Regards,

    Jack


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Friday, May 24, 2019 5:29 AM
    Moderator
  • Snippets you requested.

    public static Dictionary<string, DateTime> RecordUpdates;

    public static string DirApp = "DirUtil";

    public static ConcurrentQueue<string> RecordAdds;

    public static void UpdateEntity(string EPath, DateTime ModDate, string AppName, string AppUser)
            {
                string AppDate = DateTime.Now.ToString();
                string command_text = "UPDATE Entities SET FileUpdatedDate='@ModDate', UpdatedBy ='@AppUser', UpdateDate ='@IDate', UpdateApp ='@Appname' WHERE EPath='@EPath'";
                command_text = command_text.Replace("@AppUser", AppUser);
                command_text = command_text.Replace("@EPath", EPath);
                command_text = command_text.Replace("@UpdateApp", AppName);
                command_text = command_text.Replace("@ModDate", ModDate.ToString());
                command_text = command_text.Replace("@IDate", AppDate.ToString());
                SqlConnection CNect = new SqlConnection(RegistryConnection.ToString());
                try
                {
                    using (SqlCommand update_command = new SqlCommand(command_text, CNect))
                    {
                        CNect.Open();
                        update_command.CommandTimeout = 90;
                        try
                        {
                            int rows = update_command.ExecuteNonQuery();
                        }
                        catch (Exception exc)
                        {
                            string eMessage = "Exception caught in handling of " + EPath.ToString() + " in UpdateEntity: " + exc.Message.ToString();
                            MessageHandler("ERROR", "UpdateEntity SQL Exception", eMessage.ToString(), "UpdateEntity", AppUser);
                        }
                        CNect.Close();
                    }
                }
                catch (Exception exc)
                {
                    string eMessage = "Exception caught in handling of " + EPath.ToString() + " in UpdateEntity: " + exc.Message.ToString();
                    MessageHandler("ERROR", "UpdateEntity SQL Exception", eMessage.ToString(), "UpdateEntity", AppUser);
                }
            }

    Tuesday, May 28, 2019 6:40 PM
  • Hi

    Thank you for giving the information.

    Based on my test, I could not reproduce your problem. Your code works well. But I modify some code to let it work better. You could look at the following code.

     class Program
        {
            public static Dictionary<string, DateTime> RecordUpdates = new Dictionary<string, DateTime>();
    
            public static string DirApp = "DirUtil";
    
            public static ConcurrentQueue<string> RecordAdds=new ConcurrentQueue<string>();
            static void Main(string[] args)
            {
                Dictionary<string, string> left = new Dictionary<string, string>();
                left.Add("greet","1996-02-10");
                Dictionary<string, string> right = new Dictionary<string, string>();
                right.Add("greet", "2010-02-15");
                string appuser = "65";
                DictionaryComp(left, right, appuser);
                Console.ReadKey();
    
            }
            public static void UpdateEntity(string EPath, DateTime ModDate, string AppName, string AppUser)
            {
                string AppDate = DateTime.Now.ToString();
                string command_text = "UPDATE Entity SET FileUpdatedDate='@ModDate', UpdatedBy ='@AppUser', UpdateDate ='@IDate', UpdateApp ='@Appname' WHERE EPath='@EPath'";
                command_text = command_text.Replace("@AppUser", AppUser);
                command_text = command_text.Replace("@EPath", EPath);
                command_text = command_text.Replace("@Appname", AppName);
                command_text = command_text.Replace("@ModDate", ModDate.ToString());
                command_text = command_text.Replace("@IDate", AppDate.ToString());
                string con = @"Data Source=(localdb)\MSSQLLocalDB;Initial Catalog=Customer;Integrated Security=True;Connect Timeout=30;Encrypt=False;TrustServerCertificate=False;ApplicationIntent=ReadWrite;MultiSubnetFailover=False";
                SqlConnection CNect = new SqlConnection(con);
                try
                {
                    using (SqlCommand update_command = new SqlCommand(command_text, CNect))
                    {
                        CNect.Open();
                        update_command.CommandTimeout = 90;
                        try
                        {
                            int rows = update_command.ExecuteNonQuery();
                        }
                        catch (Exception exc)
                        {
                            string eMessage = "Exception caught in handling of " + EPath.ToString() + " in UpdateEntity: " + exc.Message.ToString();
                            //MessageHandler("ERROR", "UpdateEntity SQL Exception", eMessage.ToString(), "UpdateEntity", AppUser);
                        }
                        CNect.Close();
                    }
                }
                catch (Exception exc)
                {
                    string eMessage = "Exception caught in handling of " + EPath.ToString() + " in UpdateEntity: " + exc.Message.ToString();
                    //MessageHandler("ERROR", "UpdateEntity SQL Exception", eMessage.ToString(), "UpdateEntity", AppUser);
                }
            }
            public static void DictionaryComp(Dictionary<string, string> LeftSide, Dictionary<string, string> RightSide, string AppUser)
            {
              
                foreach (KeyValuePair<string, string> Items in RightSide)
                {
                    if (Items.Key != null)
                    {
                        if (LeftSide.TryGetValue(Items.Key.ToString(), out string CompValue))
                        {
                            DateTime LeftDate = DateTime.Parse(Items.Value, null, DateTimeStyles.RoundtripKind);
                            DateTime RightDate = DateTime.Parse(CompValue, null, DateTimeStyles.RoundtripKind);
                            int CompResult = DateTime.Compare(LeftDate, RightDate);
                            if (CompResult == 1)
                            {
                                if (!RecordUpdates.ContainsKey(Items.Key))
                                {
                                    RecordUpdates.Add(Items.Key, LeftDate);
                                    UpdateEntity(Items.Key, LeftDate, DirApp, AppUser);
                                }
                            }
                        }
                        else
                        {
                            RecordAdds.Enqueue(Items.Key);
                        }
                    }
                }
    
            }
        }

    Result:

    Best Regards,

    Jack


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Wednesday, May 29, 2019 2:54 AM
    Moderator
  • Please use the Insert Code Block tool in the editor to post code so we can read it.

    None of the code you posted calls the DictionaryComp method you said is failing. Where is this method actually called from? As Tim mentioned, the most likely issue is that you're trying to compare dictionaries that are being updated. Based upon the code you posted I could see if you passed RecordsUpdate to that method then it will fail because you're mucking with the data in question.

    Comparers should generally be self-sustaining so you shouldn't be referencing anything outside what you're given unless they are data provided to the comparer instance. Given that you're using static I suspect this is a bunch of static fields you're referencing which is going to make it really hard to manage properly, as you are seeing. Consider using a regular class to wrap the comparison logic (or consider implementing IEqualityComparer or equivalent) to isolate the data that you need. This will make it easier to confirm the correct behavior.


    Michael Taylor http://www.michaeltaylorp3.net

    Wednesday, May 29, 2019 1:45 PM
    Moderator
  • Thanks for the coding suggestion I will take it into consideration with my counterparts.
    Thursday, May 30, 2019 1:23 PM
  • Thanks for pointing out the Code Block tool.

    Below is a snippet of the core portion of the code that should give you an idea how the method in question is called.  as you can see the Dictionaries being compared are by no means being processed by any other process.  The System Comparison Dictionary (SysComp) is constructed in two steps in an attempt to speed up its construction.  The longest running method is ProcessDirectory() taking over an hour if run using a network path and less than 10 minutes if run locally.  The intermittent errors  appear to occur more often when run locally than utilizing a network path and if the executable is run in succession.  

    int DBSwitch = LoadDBComp(QPath);//Loads stored entity (Directory and File) path information to include last modified date into the Database Comparison dictionary (DBComp)
                    //
                    if (DBSwitch == 1)
                    {
                        MessageLine = "Loading of the Data Base Comparison Dictionary Finished with " + DBComp.Count.ToString() + " Keys. Taking " + ETime.Elapsed.ToString();
                        MessageHandler("INFO", "Main Progress", MessageLine.ToString(), "Main", AppUser);
                        MessageLine = "Processing Directory";
                        MessageHandler("INFO", "Main Progress", MessageLine.ToString(), "Main", AppUser);
                        //
                        Stopwatch pTime = Stopwatch.StartNew();
                        ProcessDirectory(QPath);//Walks through the Path capturing Directories and File Paths into the System Dictionary (SysComp)
                        pTime.Stop();
                        MessageLine = "Processing Directory completed with a processing time of " + pTime.Elapsed.TotalMinutes.ToString() + " Minutes. ";
                        MessageHandler("INFO", "Main Progress", MessageLine.ToString(), "Main", AppUser);
                        MessageLine = "Directory Walk of " + QPath.ToString() + " has started " + DateTime.Now.ToString();
                        MessageHandler("INFO", "Main Progress", MessageLine.ToString(), "Main", AppUser);
                        Stopwatch wTime = Stopwatch.StartNew();
                        WalkDictionary();//Using the System Directory (SysComp) gathers the last modified date of each entity
                        MessageLine = "Directory Walk of " + QPath.ToString() + " has Finished with a processing time of " + wTime.Elapsed.TotalMinutes.ToString() + " minutes.";
                        MessageHandler("INFO", "Main Progress", MessageLine.ToString(), "Main", AppUser);
                        wTime.Stop();
                        MessageLine = "Dictionary Comparison Started.";
                        MessageHandler("INFO", "Main Progress", MessageLine.ToString(), "Main", AppUser);
                        //
                        RecordAdds = new ConcurrentQueue<string>();
                        RecordUpdates = new Dictionary<string, DateTime>();
                        RecordLimbo = new List<string>();
                        //
                        Stopwatch dcTime = Stopwatch.StartNew();
                        DictionaryComp(DBComp, SysComp, AppUser);
                        MessageLine = "Dictionary Comparison has Finished with a processing time of " + dcTime.Elapsed.TotalMinutes.ToString() + " minutes. With a Total of " + RecordAdds.Count.ToString() + " for Record Additions and " + RecordUpdates.Count.ToString() + " Records Updated.";
                        dcTime.Stop();

    Thursday, May 30, 2019 1:44 PM
  • The exception is occurring in MoveNext. That means it is triggering when you're enumerating the dictionary via IEnumerator. The only places you do that is in the foreach loop and TryGetValue. I assume the foreach is the cause since you highlighted it. Therefore `RightSide` is the problem parameter. In the calling code that maps to `SysComp`. It is this dictionary that is changing. If you put a breakpoint on all lines that update this field/property I suspect you'll find it is firing (sometimes) while you're enumerating.

    But we can easily prove this is the problem by eliminating it from the picture. Before the foreach cache a copy of the dictionary. This will be expensive but it will confirm the issue is with the `SysComp` variable.

    var copy = RightSide.ToArray();
    foreach (var items in copy)
    {
     ...
    }
    This eliminates any issues with other code writing to the dictionary during enumeration. After you've confirmed this to be the issue then you can start refactoring your code to ensure that `SysComp` isn't written to while this method is called.


    Michael Taylor http://www.michaeltaylorp3.net

    Thursday, May 30, 2019 1:53 PM
    Moderator
  • Thanks for the suggestion! I am going to give it a "shot" and will post (after some testing) my results.

    As an aside the System Comparison (SysComp) dictionary is my "problem child" so I would not be surprised that there may be some "residual" process hanging out there.  What has been suspected is a Parallel foreach I use to traverse through the drive path as the "initial" pass to build SysComp.

    Thursday, May 30, 2019 3:22 PM
  • If you find there is a race condition on the dictionary then you could wrap all references to that dictionary in a lock statement as a quick workaround.  Performance would suffer but it is a quick and dirty solution.

    Alternatively use ConcurrentDictionary but note that it still doesn't support concurrent enumeration. You'd have to fetch all the keys and then attempt to retrieve the values, which may or may not exist.


    Michael Taylor http://www.michaeltaylorp3.net

    Thursday, May 30, 2019 3:58 PM
    Moderator
  • The suggestion Michael made has revealed that I may have a null key in my SysComp Dictionary.  So i am revisiting how that particular Dictionary is compiled.
    Thursday, May 30, 2019 4:56 PM
  • This is the core portion of the code that compiles the SysComp Dictionary.  I am struggling with how null values for the keys could occur.  The ProcessDirectory method is expecting a properly formatted path string to build the initial dictionary while the WalkDictionary utilizes the initial dictionary to gather last written dates for the comparison against the contents of the related database tables.  I broke this process into two steps in attempt to speed them up. ProcessDirectory runs before WalkDictionary. Code follows.
            public static Dictionary<string, List<string>> SysDir;
            public static Dictionary<string, string> SysComp;
    
            public static void ProcessDirectory(string targetDirectory)
            {
                List<string> iFile = new List<string>();
                string ELine = "";
                // Process the list of files found in the directory
                NumODirs++;
                try
                {
                    string[] fileEntries = Directory.GetFiles(targetDirectory);
    
                    foreach (string fileName in fileEntries)
                    {
                        NumOFile++;
                        string fPath = "";
                        fPath = fileName.Replace(DriveLetter, FullShare.ToString());
                        iFile.Add(fPath);
                    }
                }
                catch (PathTooLongException DirExpt)
                {
                    ELine = "PathTooLongException Directory :" + targetDirectory.ToString() + " directory length. ";
                    ELine = DirExpt.Message.ToString();
                    string[] pathD = DItems(targetDirectory.ToString()).ToArray();
                    foreach (string fileName in pathD)
                    {
                        NumOFile++;
                        string fPath = "";
                        fPath = fileName.Replace(DriveLetter, FullShare.ToString());
                        iFile.Add(fPath);
                    }
    
                }
                catch (DirectoryNotFoundException DirExpt)
                {
                    ELine = "DirectoryNotFoundException Directory :" + targetDirectory.ToString() + " directory length. ";
                    ELine = DirExpt.Message.ToString();
                    string[] pathD = DItems(targetDirectory.ToString()).ToArray();
                    foreach (string fileName in pathD)
                    {
                        NumOFile++;
                        string fPath = "";
                        fPath = fileName.Replace(DriveLetter, FullShare.ToString());
                        iFile.Add(fPath);
                    }
    
                }
                catch (Exception expt)
                {
                    ELine = "Exception in Processing Directory :" + targetDirectory.ToString() + ". ";
                    MessageHandler("ERROR", "Exception Location", ELine, "ProcessDirectory", AppUser);
                    ELine = expt.Message.ToString();
                    MessageHandler("ERROR", "Exception Detail", ELine, "ProcessDirectory", AppUser);
                }
    
                string TDir = targetDirectory.Replace(DriveLetter, FullShare.ToString());
    
                if (iFile.Count > 0)
                {
                    if (!SysDir.ContainsKey(TDir)) { SysDir.Add(TDir, iFile); }
                }
    
                // Recurse into subdirectories of this directory
                List<string> subdirectoryEntries = new List<string>();
    
                try
                {
                    subdirectoryEntries = Directory.GetDirectories(targetDirectory).ToList();
                }
                catch (PathTooLongException subExcpt)
                {
                    ELine = "Processing SubDirectories :" + subExcpt.Message.ToString() + ". ";
                    subdirectoryEntries = SItems(targetDirectory);
                }
                catch (DirectoryNotFoundException subExcpt)
                {
                    ELine = "Processing SubDirectories :" + subExcpt.Message.ToString() + ". ";
                    subdirectoryEntries = SItems(targetDirectory);
                }
                catch (Exception exft)
                {
                    ELine = "Error in Process Directory :" + targetDirectory.ToString() + " subdirectory processing.";
                    MessageHandler("ERROR", "Exception Location", ELine, "ProcessDirectory", AppUser);
                    MessageHandler("ERROR", "Exception Detail", exft.Message, "ProcessDirectory", AppUser);
                }
    
                foreach (string subdirectory in subdirectoryEntries)
                {
                    ProcessDirectory(subdirectory);
                }
    
                iFile.Clear();
            }
            //
            public static void WalkDictionary()
            {
                Parallel.ForEach(SysDir, Items =>
                {
                    if (Items.Key != null) {
                        string truePath = Items.Key.ToString();
                        if (Locale.Equals('L'))
                        {
                            string NeuVal = truePath.Replace("\\\\", "\\\\.\\");
                            truePath = NeuVal;
                        }
                        try
                        {
                            string DLUpDate = File.GetLastWriteTime(truePath).ToString();
                            SysComp.Add(Items.Key, DLUpDate);
                            if (Items.Value.Count > 0)
                            {
                                foreach (string itFile in Items.Value)
                                {
                                    string FLUpDate = File.GetLastWriteTime(itFile).ToString();
                                    SysComp.Add(itFile, FLUpDate);
                                }
                            }
    
                        }
                        catch (Exception expt)
                        {
                            string EMessage = "Entity: " + Items.Key.ToString();
                            MessageHandler("ERROR", "Entity Error", EMessage.ToString(), "WalkDictionary", AppUser);
                            EMessage = expt.Message.ToString();
                            MessageHandler("ERROR", "Entity Error Detail", EMessage.ToString(), "WalkDictionary", AppUser);
                        }
                    }
                }
                );
            }
     

    Tuesday, June 4, 2019 1:21 PM
  • Given WalkDictionary, you can get a null key if either SysDir has a key that is null or if it has any items that are null. So I think you need to step back and look at how SysDir is being populated.

    Be aware that you're exposing these things publicly so any other code may reference them as well. Unless they need to be public then make them private.

    Also, you seem to be using a lot of public properties in the code. If you don't need things to be public then don't. This just adds to the frailty of the code. You should start with local variables and then promote to fields if you need the data outside the method (and cannot use a return value). Then you promote a field to a property if you need it outside the containing class.

    Looking at your ProcessDirectory logic I don't understand the catch logic. You start the function by trying to get all the files in it. For each file you replace the drive letter with a file share name and then add it to a list. You're counting the files as well but that seems redundant because you can always get the final count of files at the end (`iFile.Count`).

    If the path is too long then you write to some ELine variable but note that in each case you write a value and then stomp over that value with a new message. The first write to ELine does nothing. After that you call DItems to get a new list of flies and then repeat the same logic you had earlier. So DItems somehow handles these errors? Maybe something simpler, like this (not tested).

    public static void ProcessDirectory ( string targetDirectory )
    {
        var iFile = new List<string>();
                
        // Process the list of files found in the directory
        NumODirs++;
        try
        {                
            var files = TryGetFiles(targetDirectory);
            iFile.AddRange(files);                
        } catch (Exception expt)
        {
            var msg = "Exception in Processing Directory :" + targetDirectory + ". ";
            MessageHandler("ERROR", "Exception Location", msg, "ProcessDirectory", AppUser);
    
            msg = expt.Message.ToString();
            MessageHandler("ERROR", "Exception Detail", msg, "ProcessDirectory", AppUser);
    
            //Note that at this point you have no files so do you really want to continue?
        };
    
        string TDir = UseFileShare(targetDirectory);
    
        if (iFile.Count > 0)
        {
            if (!SysDir.ContainsKey(TDir))
            { SysDir.Add(TDir, iFile); }
        }
    
        // Recurse into subdirectories of this directory            
        try
        {
            var subdirectoryEntries = TryGetDirectories(targetDirectory).ToList();
    
            foreach (var subdirectory in subdirectoryEntries)
                ProcessDirectory(subdirectory);
        } catch (Exception e)
        {
            var msg = "Error in Process Directory :" + targetDirectory.ToString() + " subdirectory processing.";
            MessageHandler("ERROR", "Exception Location", msg, "ProcessDirectory", AppUser);
            MessageHandler("ERROR", "Exception Detail", e.Message, "ProcessDirectory", AppUser);
        }            
    
        NumOFile += iFile.Count;
    }
    
    private static IEnumerable<string> TryGetDirectories ( string targetDirectory )
    {
        try
        {
            return Directory.GetDirectories(targetDirectory);
        } catch (PathTooLongException e)
        {
            return SItems(targetDirectory);
        } catch (DirectoryNotFoundException e)
        {
            return SItems(targetDirectory);
        };
    }
    
    private static IEnumerable<string> TryGetFiles ( string targetDirectory )
    {
        try
        {
            //Using GetFiles which will trigger an immediate enumeration here
            return from f in Directory.GetFiles(targetDirectory)
                    select UseFileShare(f);
        } catch (PathTooLongException)
        {
            return from f in DItems(targetDirectory).ToArray();
        } catch (DirectoryNotFoundException)
        {
            return from f in DItems(targetDirectory).ToArray();                
        };
    }
    
    private static string UseFileShare ( string targetDirectory )
    {
        return targetDirectory.Replace(DriveLetter, FullShare.ToString());
    }


    Michael Taylor http://www.michaeltaylorp3.net

    • Marked as answer by PaulFlores Thursday, September 19, 2019 1:37 PM
    Tuesday, June 4, 2019 2:00 PM
    Moderator
  • Michael,

    Thanks for voicing your concerns but the code has been "sanitized" quite a bit. 

    Thanks once again for your suggestions and will pass them along.  I tend to agree that a less convoluted approach may be needed and to some extent "sanitizing" the code to post may have introduced inconsistencies.

    Tuesday, June 4, 2019 5:19 PM
  • Hi PaulFlores,

    I want to confirm if your problem has been solved. If so, please post "Mark as answer" to the appropriate answer, so that it will help other members to find the solution quickly if they face a similar issue. If not, please feel free to let us know.

    Best Regards,

    Jack


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Wednesday, June 5, 2019 1:52 AM
    Moderator
  • The culprit turned out to be some interesting edge cases that testing turned up.  The most problematic edge case would seem to be most obvious dealing with the root of the hierarchy that the folder / directory structure represents.

    Thanks to Mike for his suggestion they were helpful in tracking down the edge cases.

    Marking this answered!

    • Marked as answer by PaulFlores Wednesday, June 12, 2019 9:56 PM
    Wednesday, June 12, 2019 9:56 PM
  • Hi Paul,

    >>Thanks to Mike for his suggestion they were helpful in tracking down the edge cases.<<

    Maybe you should mark his suggestion as an Answer too. It'll make it easier for people to know which reply you're talking about.  =0)


    ~~Bonnie DeWitt [C# MVP]

    http://geek-goddess-bonnie.blogspot.com


    Thursday, June 13, 2019 4:50 AM
    Moderator