SLQ Database Application method RRS feed

  • Question

  • Hi guys,

    I have about 15 years programming experience but I'm an SQL / data base beginner.

    I am writing an application in VS 2008 .NET C#. The primary function of this application is to receive high frequency data  (on the order of a millisecond) on a namedpipe, and store the data into a SQL Local Database.

    To start, I have the named pipe event driven portion of my app working. I am now trying to add the database.

    A few consideration I believe are important. 

    1. The database has only 1 table, with no relationships.  I think of it as a very large csv file, with the most recent data at the top of the file.
    2. Each row has 5 elements, of the following types (numeric(19,4), nchar(7), nchar(3), real, numeric(9,0)
    3. This application will collect data all day, every week day.  The database file will become very large. 

    A few questions.

    1. I have SQL 2008.  Should I use this program to create a database file, or is the VS 2008 built in ability sufficient?  I would rather keep everything in VS 2008 if possible.
    2. I like the method of accessing this database using LINQ, but I'm not sure if that will be a problem because the database will grow to be very large.  Does LINQ load an entire table into memory, or can you limit how much data to load?
    Any ideas or concerns you can think of? 

    Thank you sincerely for you assistance.


    Friday, March 19, 2010 1:57 PM


  • LINQ does not load an entire table into memory. It does lazy loading i.e. load on a need-to-have basis. So you are technically OK to use LINQ. But I personally believe L2S is an overkill for your scenario. I mean you dont have a Domain Model at all. It doesn't warrant a EF or L2S methodology. Plain old SqlCommand and ExecuteNonQuery should work for you just fine.

    As a side statement, I would probably think of collecting data into memory and dump this data into SQL as one big chunk every minute or so. As you probably know already, MS SQL is very good at handling bulk DML ops.



    Friday, March 19, 2010 8:12 PM