none
Replacing chars in a LIST<T> RRS feed

  • Question

  • Hello,

    Currently I have a list of object "LIST<class>" which I need to replace illegal chars (for instance semicolumn ";") before generating the CSV. However the following code it's taking 3x times more than without it:

    public static void ListToCsv<T, V>(List<V> query, string path)
            {
              var propertiesDto = typeof(T).GetProperties().Select(x => x.Name).ToList();
                var propertiesIQueryable = typeof(V).GetProperties().Where(x => propertiesDto.Contains(x.Name));
                
                using (StreamWriter sw = new StreamWriter(path))
                {
                    sw.WriteLine(string.Join(UniversalVariables.csvDelimiter.ToString(), propertiesDto.Select(p => p)));
    
                    foreach (var row in query)
                    {
                        var obj = Activator.CreateInstance<T>();
                        string line = "";
    
                        foreach (var propertyQuery in propertiesIQueryable)
                        {
                            var value = RemoveIllegalChar(propertyQuery.GetValue(row, null));
        //without the above line is 3x faster                        
        //var value = propertyQuery.GetValue(row, null);
                            line += value + UniversalVariables.csvDelimiter.ToString();
                        }
    
                        sw.WriteLine(line.Remove(line.Length - 1));
                    }
                   
                }
    }
    
    private static string RemoveIllegalChar(object value)
            {
    
                var newValue = value.ToString();
    
                if (value.GetType() == Type.GetType("System.String"))
                {
                    // Updating date - time to get a valid .csv file:
                    newValue += newValue.Replace(";", ",").Replace(@"\t|\n|\r", "");
                };
    
                return newValue;
            }

    Before this method I have a LINQ query which maybe I could use to replace the chars, but I was not able to do it as my List turns into a List<string> instead of a List<class>:

    var lstRecords = (from cp in dbLinq_2.Table
                     select cp).ToList();

    Thursday, February 27, 2020 7:17 AM

All replies

  • Maybe, instead of altering the data, you can generate a CSV that includes quotation marks, for example:

    "field1","field2, containing spaces and comma","quotation ""marks"""

    Thursday, February 27, 2020 8:25 AM
  • Indeed Viorel, this is a option and in the worst case scenario I will use it. The negative point is the size as I have a huge amount of records and columns (more than 20 million rows) and as you know It will increase the file size. 
    Thursday, February 27, 2020 8:39 AM
  • Hi FcabralJ,
    You can try to use StringBuilder or Regex.Replace.
    Here is a link about discussing the faster way to replacie multiple characters in a string.
    [Replacing multiple characters in a string, the fastest way?]
    Hope it could be helpful for you.
    Best Regards,
    Daniel Zhang


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Friday, February 28, 2020 7:52 AM
  • Hello friend in the function RemoveIllegalChar change changing replace may worked

    newValue += newValue.Replace(";", ",").Replace(@"\t", "").Replace(@"\n","").Replace(@"\r","");

    Friday, February 28, 2020 2:32 PM
  • Hi Daniel,

    I've tried to implement the Regex which was the fastest way but I didn't get good performance. Maybe my problem is more related to the reflection than the remove itself. 

    foreach (var propertyQuery in propertiesIQueryable)
    {
                            
      var value = RemoveIllegalChar(propertyQuery.GetValue(row, null));
    }


    private static string RemoveIllegalChar(object value) { if (value == null) { return ""; } char[] badChars = { ';', '\t', '\n', '\r' }; int replChar = value.ToString().IndexOfAny(badChars); if (replChar < 0) { return value.ToString(); } else { Regex reg = new Regex("\t|\n|\r|;"); MatchEvaluator eval = match => { switch (match.Value) { case ";": return ""; case "\t": return ""; case "\n": return ""; case "\r": return ""; default: throw new Exception("Unexpected match!"); } }; return reg.Replace(value.ToString(), eval); } }

    Monday, March 2, 2020 5:42 AM