locked
Problem with import of .csv files RRS feed

  • Question

  • User256135728 posted

    Hello everyone,

    In advance, please excuse any faults I make in this post.

    So, to the point. My asp.net application has to import .csv files and display data from those files. The files are continually updated every 30 seconds or so. I have used the Microsoft JET OLE DB Data Provider for the parsing of those csv files.

    It is also important to tell you that I am using an asynchronous http handler to perform this operation as there is a large number of concurrent visitors. So basically this is what they call a "comet" application, also known as the long polling approach.

    So, I am using a system file watcher object to monitor the source folder for changes, and once the ftp uploader uploads the new data, the file system watcher fires an on changed event which in turn signals my asynchronous handler that it can resume with the response, meaning it can read the new data and send a response with fresh data to the client.

    However, I am receiving these errors constantly and they are crashing my application:

    Event Type: Error
    Event Source: ASP.NET 4.0.30319.0
    Event Category: None
    Event ID: 1325
    Date: 20/04/2011
    Time: 15:33:14
    User: N/A
    Computer: xxx
    Description:An unhandled exception occurred and the process was terminated.

    Application ID: /LM/W3SVC/1/ROOT
    Process ID: 5264
    Exception: System.Data.OleDb.OleDbException
    Message: Unspecified error

    StackTrace:    
    at System.Data.OleDb.OleDbConnectionInternal..ctor(OleDbConnectionString constr, OleDbConnection connection) 
    at System.Data.OleDb.OleDbConnectionFactory.CreateConnection(DbConnectionOptions options, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningObject)   
    at System.Data.ProviderBase.DbConnectionFactory.CreateNonPooledConnection(DbConnection owningConnection, DbConnectionPoolGroup poolGroup)   
    at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)   
    at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)   
    at System.Data.OleDb.OleDbConnection.Open()   
    at Broker.brCSV.readCSV(String fileName)   
    at SwatchTiming.AsynchOperation.StartAsyncTask(Object workItemState)   
    at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state) 
    at System.Threading.ExecutionContext.runTryCode(Object userData)   
    at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)   
    at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)   
    at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx) at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()   
    at System.Threading.ThreadPoolWorkQueue.Dispatch()   
    at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

     

     

    Event Type: Error
    Event Source: .NET Runtime 4.0 Error Reporting
    Event Category: None
    Event ID: 5000
    Date: 20/04/2011
    Time: 15:33:14
    User: N/AComputer: xxx
    Description:EventType clr20r3, P1 w3wp.exe, P2 6.0.3790.3959, P3 45d6968e, P4 system.data, P5 4.0.0.0, P6 4ba1e064, P7 1ea3, P8 87, P9 system.data.oledb.oledbexception, P10 NIL.
    For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.Data:0000: 63 00 6c 00 72 00 32 00   c.l.r.2.0008: 30 00 72 00 33 00 2c 00   0.r.3.,.0010: 20 00 77 00 33 00 77 00    .w.3.w.0018: 70 00 2e 00 65 00 78 00   p...e.x.0020: 65 00 2c 00 20 00 36 00   e.,. .6.0028: 2e 00 30 00 2e 00 33 00   ..0...3.0030: 37 00 39 00 30 00 2e 00   7.9.0...0038: 33 00 39 00 35 00 39 00   3.9.5.9.0040: 2c 00 20 00 34 00 35 00   ,. .4.5.0048: 64 00 36 00 39 00 36 00   d.6.9.6.0050: 38 00 65 00 2c 00 20 00   8.e.,. .0058: 73 00 79 00 73 00 74 00   s.y.s.t.0060: 65 00 6d 00 2e 00 64 00   e.m...d.0068: 61 00 74 00 61 00 2c 00   a.t.a.,.0070: 20 00 34 00 2e 00 30 00    .4...0.0078: 2e 00 30 00 2e 00 30 00   ..0...0.0080: 2c 00 20 00 34 00 62 00   ,. .4.b.0088: 61 00 31 00 65 00 30 00   a.1.e.0.0090: 36 00 34 00 2c 00 20 00   6.4.,. .0098: 31 00 65 00 61 00 33 00   1.e.a.3.00a0: 2c 00 20 00 38 00 37 00   ,. .8.7.00a8: 2c 00 20 00 73 00 79 00   ,. .s.y.00b0: 73 00 74 00 65 00 6d 00   s.t.e.m.00b8: 2e 00 64 00 61 00 74 00   ..d.a.t.00c0: 61 00 2e 00 6f 00 6c 00   a...o.l.00c8: 65 00 64 00 62 00 2e 00   e.d.b...00d0: 6f 00 6c 00 65 00 64 00   o.l.e.d.00d8: 62 00 65 00 78 00 63 00   b.e.x.c.00e0: 65 00 70 00 74 00 69 00   e.p.t.i.00e8: 6f 00 6e 00 20 00 4e 00   o.n. .N.00f0: 49 00 4c 00 0d 00 0a 00   I.L.....

     

    and...

     

    Event Type: Error
    Event Source: .NET Runtime
    Event Category: None
    Event ID: 1026
    Date: 20/04/2011
    Time: 15:34:26
    User: N/A
    Computer: xxx
    Description:Application: w3wp.exe
    Framework Version: v4.0.30319
    Description: The process was terminated due to an unhandled exception.Exception
    Info: System.Data.OleDb.OleDbException

    Stack:   
    at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(System.Data.Common.DbConnection, System.Data.ProviderBase.DbConnectionFactory)   
    at System.Data.OleDb.OleDbConnection.Open()   
    at Broker.brCSV.readCSV(System.String)   
    at [ProjectNamespace].AsynchOperation.StartAsyncTask(System.Object)   
    at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(System.Object)   
    at System.Threading.ExecutionContext.runTryCode(System.Object)   
    at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode, CleanupCode, System.Object)   
    at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)   
    at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)   
    at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()   
    at System.Threading.ThreadPoolWorkQueue.Dispatch()   
    at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()

    For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

    At first I thought all this has something to do with the connection to the .csv file, but since I've changed the access mode in the connection string, and tried all the possible solutions (readOnly, Mode=1, readwrite, Mode=3, Mode=16, whatever) and did not fix it, I am thinking that even though it says OleDbException that it has something to do with something else.

     

    Does anybody have any idea on what the hell is the problem?

     

    Thank you very much everybody...

    Wednesday, April 20, 2011 10:48 AM

All replies

  • User269602965 posted

    Load the CSV files into a database with C# or VB.NET.

    Have the users access the database table instead of the CSV file.

    With connection pooling, etc., the database will be able to handle many more users.

    Thursday, April 21, 2011 7:04 PM
  • User256135728 posted

    Well, yes, I could have done that, but that was not the request from the client, and even if it was that would require changing the whole structure of the application for which I do not have time now.

    Thank you for your reply though :) 

    Friday, April 22, 2011 1:56 PM