locked
Using CopyTo method from FileStream to a MemoryStream RRS feed

  • Question

  • Hi,

    Yesterday, I stumbled across the new CopyTo method and I decided to try it to see if it could shorten the upload of files in my application. So I compared these two approaches.

    First approach :

    FileStream fsFile = new FileStream(file_name, FileMode.Open, FileAccess.Read, FileShare.Read, 4096, false);
    BinaryReader  reader = new BinaryReader(fsFile);

    Second approach :

    FileStream fsFile = new FileStream(file_name, FileMode.Open, FileAccess.Read, FileShare.Read, 4096, false); 
    MemoryStream fsFast = new MemoryStream((int)fsFile.Length);
    fsFile.CopyTo(fsFast, 4096);
    fsFile.Close();
    fsFast.Position = 0;
    BinaryReader  reader = new BinaryReader(fsFast);

    I do this reading for several files in parallel, and reading their content takes about the same time as copying the files from the network to my local disk using Windows Explorer. Still, I find it odd that both approaches take exactly the same time to complete the reading operations (40s for 364mb). I would have though that copying the file to memory would make it much faster to read one byte at a time. Apparently it is not the case! Also, in the second approach, the CPU use is a little bit higher, but nowhere near what I would have expected. My knowledge of the underlying processes might be incomplete... Maybe some buffering occurs each time I call ReadByte?

    So... is something wrong with my experiment? Was is to be expected? As any of you have found faster ways to read binary files?

    Thanks,
    Oliver
    Thursday, January 28, 2010 3:16 PM

Answers

  • Hi Oliver,

    Off the top of my head, I'm not entire sure why there isn't a difference.  Note that FileStream does do buffering, but it doesn’t buffer the entire file into memory – it does it in chunks as it is being read.

    In either case, you still need to use FileStream to read the contents of the file.  I'd recommend just sticking with FileStream instead of buffering the entire contents of the file into a MemoryStream and then reading from the MemoryStream.  Otherwise, if you end up trying to read a very large file, it is possible that you will run out of memory.

    Thanks,

    Justin Van Patten
    Program Manager
    Base Class Libraries

    Thursday, January 28, 2010 6:36 PM

All replies

  • Hi Oliver,

    Off the top of my head, I'm not entire sure why there isn't a difference.  Note that FileStream does do buffering, but it doesn’t buffer the entire file into memory – it does it in chunks as it is being read.

    In either case, you still need to use FileStream to read the contents of the file.  I'd recommend just sticking with FileStream instead of buffering the entire contents of the file into a MemoryStream and then reading from the MemoryStream.  Otherwise, if you end up trying to read a very large file, it is possible that you will run out of memory.

    Thanks,

    Justin Van Patten
    Program Manager
    Base Class Libraries

    Thursday, January 28, 2010 6:36 PM
  • Thanks Justin.

    Indeed, if there are no gain in speed, I should switch back to using only FileStream. I'll keep searching for a way to increase the speed for a couple of hours, just in case I find something...

    Have a nice day,
    Oliver
    Friday, January 29, 2010 2:52 PM