handle large files

Question

User-1982615125 posted

how can i handle large files by reading the data that i need and saving the changes only. so avoid reading and writing everything.

as ive seen an infinite of times by other posts:

I AM NOT LOOKING FOR BUFFERING/CASHING. IM LOOKING FOR SPEEDING UP ACCES TO THE FILE. SINCE THE LIFESPAN OF THE APP IS LESS THAN A SECOND. this is why the speed has to be found in the reading and writing, rather than buffering an instance of the file.

Answer 1

User475983607 posted

how can i handle large files by reading the data that i need and saving the changes only. so avoid reading and writing everything.

as ive seen an infinite of times by other posts:

I AM NOT LOOKING FOR BUFFERING/CASHING. IM LOOKING FOR SPEEDING UP ACCES TO THE FILE. SINCE THE LIFESPAN OF THE APP IS LESS THAN A SECOND. this is why the speed has to be found in the reading and writing, rather than buffering an instance of the file.

Your question infers that building something like this is common knowledge.  The truth is it requires a fair amount of Computer Engineering and knowledge of the target platform.  The common solution for us mere mortals is a making use of a database (or distributed cache) as the folks that build these services handled the technical details.   

This advice has been given freely to you over your many threads on the this subject. 

If you are interested in learning the technical details, I suggest that you visit a support forum more inline with the subject.  Or perhaps take a few CE classes. 

Answer 2

User753101303 posted

Hi,

Your description is basically what a database management system is doing already (some of them have a "in table memory feature" or maybe a https://en.wikipedia.org/wiki/Document-oriented_database or a https://en.wikipedia.org/wiki/Key-value_database would better fir your needs ?).

Historically data storage began with random acces files as shown at http://www.worldbestlearningcenter.com/index_files/csharp-file-stream-random.htm and db systems were added on top of that for additional flexibility.

Is this enough ? Do you need some https://en.wikipedia.org/wiki/ACID properties? What if you later want to add a property to the objects you are storing or if you want to store another kind of objects.