Looking for the best/fastest way to retrieve data from a CSV file RRS feed

  • Question

  • I've been programming for several years, but until now, I've never needed to access, data so it's all very new to me. I need to read data from a CSV file. A file that contains about 7000 lines if that's important. There seems to be several ways to do this. 

    1. I can use FileHelpers and iterate over each line in a <List>. But this seems slow as I'll need to do this A LOT of times. 

    2. A Co-Worker suggested I use ODBC and run a query on the file. Not a problem, but i have no idea how to do that.

    3. Use LINQ. I've only played with LINQ a little. 

    4. A combination of them...

    5. Something else I haven't heard of yet...

    Here's my question. I realize i need to do some research. But because I will be doing a lot of "querying" in this program, speed is my biggest concern. For example, I would like to be able to "select this and that from this file, where this column has this data" (pseudo code obviously). I would like to learn the fastest method. Can anyone tell me the best direction to take?

    I hope I've provided enough info. If not let me know. 

    Thank you. 

    PS. I don't want to fill a DataGridView. Good grief, that's all I can seem to find info on. lol.

    • Edited by ASI Tech Friday, December 15, 2017 10:33 PM added an example
    Friday, December 15, 2017 9:35 PM

All replies

  • The easiest for you and the computer is probably to use String.Split if it will work for you. It would not work easily if any of the fields have commas as part of the data and that is a definite possibility. If the file(s) you need to process don't have that feature then try String.Split. You can do a quickie test by just reading every record and doing a String.Split and then if any record splits into an array with more items than expected then you know there are records that make String.Split not useful.

    Sam Hobbs

    Friday, December 15, 2017 10:42 PM