none
Batch update into database RRS feed

  • Question

  • Hi there, I am parsing a log file and there's a need where I have to first

    INSERT ROW --> QUERY ROW --> UPDATE ROW

    All of the above operations happen at different times when parsing the log file. Approx. there are 15000 rows and each time its expensive if I have to INSERT (Open & Close connection), QUERY (Open & Close connection) and UPDATE (Open & Close connection).

    Is there a way where I can use something similar to a Dataset where I can work just on the Dataset (for storing without always going to database) and when parsing is completed I can directly INSERT all the 15000 rows at once into database. Memory is not at all a problem when working like this. I dont want to always open and close DB connection.
    • Moved by Harry Zhu Wednesday, July 8, 2009 4:09 AM relating to oracle data provider (From:Visual C# General)
    Tuesday, July 7, 2009 3:10 AM

All replies

  • Maybe you shed some light on what database you are using, is it on the same machine or some other computer?
    Different DB engines have different methods for importing data, usually they take it from a local hard disk.
    Please remember to mark the replies as answers if they answered your question :)
    Tuesday, July 7, 2009 11:24 AM
  • you can try using System.Data.SqlClient.SqlBulkCopy

    Is there a way where I can use something similar to a Dataset where I can work just on the Dataset (for storing without always going to database) and when parsing is completed

    - store all your data in a DataTable after your parsing is completed you can perform multiple insert in your database by using SqlBulkCopy


    SqlBulkCopy bulk = new SqlBulkCopy(ConnectionString)

    bulk.DestinationTableName = TableName
    bulk.WriteToServer(DataTable)
    Tuesday, July 7, 2009 2:28 PM
  • I use Oracle 10g and its on a Linux server and
    Does System.Data.OracleClient can handle batch updates?

    If this does not work I need to change programming lang.
    Tuesday, July 7, 2009 9:47 PM