locked
Slow file access operations in WinRT

    Question

  • I need to open a lot of files (chosen e.g. by FilePicker) and read several bytes from each of them. I was using fopen and fread operations in classic Windows, it was fast enough. But in WinRT, I am forced by SDK to use IRandomAccessStream with OpenAsync and ReadAsync methods and found out, that these are slow as hell. Is there a way to speed up things as much as possible? It is really an issue, the file open and read operations on the same set of files are now 10-40x slower, so what took 10s in desktop Win, take e.g. 5 minutes in WinRT!

    Thanks for any advice.

    Friday, December 07, 2012 7:29 AM

All replies

  • As file operations in WinRT completely asynchronous, it should not affect the overall performance of your apps. You can execute several operations concurrently.

    As far I know WinRT asynchronous operations runs in background and it relieves the UI thread so that it becomes responsive.

    Friday, December 07, 2012 3:15 PM
  • You can use Windows Performance Recorder and Windows Performance Analyzer to have a look at the I/O performance, also.  What you'll probably see is that the Windows.Storage APIs force flushing, which adds some latency to the API call.

    It's appears hard if not impossible to get around this for Windows Store apps.  I'm struggling with this on unzip performance - unzipping an archive with many small files using ZipArchive is very inefficient; I see about 6x better on the same task using Ionic.Zip in a console app.

    If you can parallelize your file operations, then you may get some speedup, as Mokarrom said.  I might try this today with my test unzip application, and I can post the results.

    Friday, December 07, 2012 4:13 PM
  • I have not problems with UI responsivity, but with file scanning speed. It runs in background, but cannot take days ;-)

    I've tried more parallel processing, but it is still cca 6x slower than in desktop. The problem is in fact, among others, that reading from a lot of files at one time is not as fast as sequential reading.

    Monday, December 10, 2012 2:14 PM
  • The big difference here is that your Windows Store app doesn't have sufficient privileges to read the files directly. The authorization to read them granted by the user picking them is indirect: an external broker process reads the files and then passes the data back to your app, so you have this extra cost for each file. Reading a small amount from many files is pretty much a worst case for this and will magnify the cost of the IPC.

    When you say you need to read some bytes from many files are these arbitrary bytes or are they standard properties? Common cases such as getting properties or thumbnails from many files can be prefetched by creating an appropriate file query. See StorageFolder.CreateFileQueryWithOptions

    --Rob

    Monday, December 10, 2012 9:09 PM
    Owner
  • Rob: thanks for the response. Unfortunately, we need to read more properties from more file formats, than this standard method supports.
    Monday, December 10, 2012 9:23 PM