How costly are new/Dispose on a DataTable? RRS feed

  • Question

  • I guess I'm asking how much it would cost to create new tables, as opposed to copying rows from a table into an array and clearing the Rows collection.

    This table has maybe 10 columns defined, each with a type, but with no constraints. In future we may add another smaller table or two & do the same things with them, but there will be no DataSet, since there's no need for this program to concern itself with constraints. It looks like we'll actually be running on .NET 3.5 (although I'm still working in VS2005 for the time being.)


    The question in the subject came up when thinking about a possible solution that would involve newing a lot of DataTable objects with the same structure, so I thought I would post the requirements/proposed solution in case you read this and see a better way to handle things.


    We want an application to read messages (multicast) from the network, collect/format the information we want to save from these messages, and store that information into a SQL database table. This thing runs 24/7.


    My thought is that this would happen on 3 threads:


    Thread1 reads the messages from the socket, deserializes them, and decides whether each is the right kind of message to pass on to Thread2. (This is the minimum Thread1 can do, because it is using code already in place to do this -- that code just delivers a deserialized message to an event handler in my code. It's important that this thread does as little as possible, otherwise we get dropped packets when volume is high.)


    Thread2 puts the info from the messages into an in memory DataTable as it gets them. Then we have a timer that queues an operation to Thread2, which sends the contents of the table over to Thread3 for writing, and clears (or replaces) the original table. The idea here is that all access to the current table take place on the same thread, so we don't have to worry about any operations on the table conflicting.


    Thread3 writes the contents of its DataTable (or DataRow array?) to the database (using SqlBulkCopy.)


    If you see any better ways to do the work of threads 2 & 3, feel free to suggest.


    Having asked, I suspect I'll have to bite the bullet and use a profiler to find out (my experience on this job suggests there's little questions that we'll be pushing this ap to its limits & performance issues will matter), but before going though that pain, I'll wait in case anyone wants to critique my design or actually has experience to know.




    Friday, April 25, 2008 4:46 PM


  • If it matters that much to you, you'll need to time it and see.


    However, it is typical that the actual writing to the database tends to use the most significant percentage of the time.



    Friday, April 25, 2008 11:57 PM