locked
File Processing - I/O RRS feed

  • Question

  • Hi, apologies for a simple C# question, am not from .Net area.


    I am looking for a C# sample program that takes  #1. a CSV file as input #2. Changes a couple of columns  #3. writes the result into a new file.


    Could you please direct me to a sample piece of code or reply here.


    many thanks for your help

    Friday, April 3, 2020 11:12 PM

All replies

  • Perhaps if you can provided exact details rather than high level details we can guide you to a solution but with what you have presented could go easily down different paths.

    And when providing exact details try to think hard and get all details down the first time, many when ask for the same information will not take the time and add details later down the road in the thread which means everyone waste their time from that point on.


    Please remember to mark the replies as answers if they help and unmarked them if they provide no help, this will help others who are looking for solutions to the same or similar problem. Contact via my Twitter (Karen Payne) or Facebook (Karen Payne) via my MSDN profile but will not answer coding question on either.

    NuGet BaseConnectionLibrary for database connections.

    StackOverFlow
    profile for Karen Payne on Stack Exchange

    Saturday, April 4, 2020 12:19 AM

  • I am looking for a C# sample program that takes  #1. a CSV file as input #2. Changes a couple of columns  #3. writes the result into a new file.


    Could you please direct me to a sample piece of code or reply here.



    A simple web search will show up countless examples of using a CSV file in C#,
    Try searching for this:

    msdn c# read csv file

    - Wayne

    Saturday, April 4, 2020 1:09 AM
  • Hi etl2016,
    You can use TextFieldParser to read the csv.file and add the text to the list. Then use StreamWriter to save the file.
    Note: Add Microsoft.VisualBasic reference into project.
    Here is a code example you can refer to.

    Csv csv = new Csv();
    csv.OpenSingleFile(@"C:\Users\Desktop\test1.csv");
    csv.FileSave(@"C:\Users\Desktop\test2.csv");

    Csv class:

    public class Csv
    {
        public List<List<string>> Rows = new List<List<string>>();
    
        public char SeparatorChar;
    
        public int MaxRowsInOneFile = 30000;
    
        public void OpenSingleFile(string filePath)
        {
            if (!File.Exists(filePath))
            {
                throw new Exception("File Not exist!");
            }
    
            using (TextFieldParser parser = new TextFieldParser(filePath))
            {
                parser.TextFieldType = FieldType.Delimited;
    
                parser.SetDelimiters(SeparatorChar.ToString());
    
                while (!parser.EndOfData)
                {
                    Rows.Add(parser.ReadFields().ToList());
                }
            }
        }
    
        public void FileSave(string path)
        {
            if (!path.ToLower().EndsWith(".csv"))
            {
                path += ".csv";
            }
    
            var correctPath = Directory.GetParent(path).ToString();
    
            Directory.CreateDirectory(correctPath);
    
            FileSave(path, false);
        }
    
        private void FileSave(string path, bool appendFile)
        {
            var files = SeparateCsvToFiles(Rows, path);
    
            foreach (var file in files)
            {
                using (StreamWriter writer = new StreamWriter(file.Key, appendFile, Encoding.UTF8))
                {
                    foreach (var row in file.Value)
                    {
                        writer.WriteLine(BuildCsvRow(row));
                    }
                }
            }
        }
    
        private List<KeyValuePair<string, List<List<string>>>> SeparateCsvToFiles(List<List<string>> rows, string originalPath)
        {
            List<List<string>> rowsCopy = new List<List<string>>(rows);
    
            var files = new List<KeyValuePair<string, List<List<string>>>>();
    
            int counter = 1;
            Func<string> newpath = () => { return originalPath.Replace(".csv", counter++ + ".csv"); };
    
            if (rowsCopy.Count < MaxRowsInOneFile)
            {
                files.Add(new KeyValuePair<string, List<List<string>>>(originalPath, rowsCopy));
            }
            else
            {
                List<List<string>> fileRows;
    
                while (rowsCopy.Count > MaxRowsInOneFile)
                {
                    fileRows = rowsCopy.GetRange(0, MaxRowsInOneFile);
                    rowsCopy.RemoveRange(0, MaxRowsInOneFile);
    
                    files.Add(new KeyValuePair<string, List<List<string>>>(newpath(), fileRows));
                }
    
                fileRows = rowsCopy.GetRange(0, rowsCopy.Count());
    
                files.Add(new KeyValuePair<string, List<List<string>>>(newpath(), fileRows));
            }
    
            return files;
        }
        public void FileAppend(string path)
        {
            FileSave(path, true);
        }
        private string BuildCsvRow(List<string> rowCells)
        {
            StringBuilder builder = new StringBuilder();
    
            bool firstColumn = true;
    
            foreach (string value in rowCells)
            {
                if (value != null)
                {
                    // Add separator if this isn't the first value
                    if (!firstColumn)
                        builder.Append(SeparatorChar);
                    // Implement special handling for values that contain comma or quote
                    // Enclose in quotes and double up any double quotes
                    if (value.IndexOfAny(new char[] { '"', SeparatorChar }) != -1)
                    {
                        builder.AppendFormat("\"{0}\"", value.Replace("\"", "\"\""));
                    }
                    else
                    {
                        builder.Append(value);
                    }
    
                    firstColumn = false;
                }
            }
            return builder.ToString();
        }
    }

    Best Regards,
    Daniel Zhang


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Thursday, April 9, 2020 9:31 AM
  • Hi etl2016,
    Has your problem been solved? If it is resolved, we suggest that you mark it as the answer. So it can help other people who have the same problem find a solution quickly.
    Best Regards,
    Daniel Zhang


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Thursday, April 23, 2020 9:09 AM
  • From https://www.dotnetperls.com/textfieldparser

    TextFieldParser is too slow

    Monday, April 27, 2020 11:50 AM
  • From https://www.dotnetperls.com/textfieldparser

    TextFieldParser is too slow

    Have you looked at using a StreamReader for reading a file? See a simple example here. Then for writing look at StreamWriter. See Microsoft docs for examples. There are plenty of examples on the web for you to put your own solution together.


    Please remember to mark the replies as answers if they help and unmarked them if they provide no help, this will help others who are looking for solutions to the same or similar problem. Contact via my Twitter (Karen Payne) or Facebook (Karen Payne) via my MSDN profile but will not answer coding question on either.

    NuGet BaseConnectionLibrary for database connections.

    StackOverFlow
    profile for Karen Payne on Stack Exchange

    Monday, April 27, 2020 12:18 PM
  • Too slow for what?  If you're reading a 300-line file, then it's certainly not too slow.  If you have a 500,000 line CSV file, then you are using the wrong format for your database.

    Tim Roberts | Driver MVP Emeritus | Providenza &amp; Boekelheide, Inc.

    Monday, April 27, 2020 10:50 PM