locked
Performance of StorageFile::GetBasicPropertiesAsync() and CRT _fstat64() API

    Question

  • My achievement in my Windows Store APP is to retrieve some file attributes like file size and last modified time.

    I tried CRT API: _fstat64() before, and it work fine.

    And then I try using Windows Runtime API: StorageFile::GetbasicPropertiesAsync(), but find that this API's performance not as fast as CRT API.

    So, I wrote a sample for comparing the performance between these 2 APIs, here's the source:

    static int s_case7_loop_count = 0;

    static uint64_t s_case7_start_time = 0;
    static uint64_t s_case7_end_time = 0;


    char s_1kb_data[1025] = "1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye 1000 btye end.";

    void MainPage::RunTestCaseCRT() { // 1. create file DWORD desiredAccess = GENERIC_READ|GENERIC_WRITE; DWORD shareMode = FILE_SHARE_DELETE | FILE_SHARE_READ | FILE_SHARE_WRITE; DWORD createDisposition = CREATE_ALWAYS;; HANDLE h32 = INVALID_HANDLE_VALUE; HANDLE hStat = INVALID_HANDLE_VALUE; int hCrt = 0; int flags = 0; int nbytes = 0; String^ filePath = ApplicationData::Current->LocalFolder->Path + "\\TestCaseCRT.txt"; CREATEFILE2_EXTENDED_PARAMETERS cf2ex = {0}; cf2ex.dwSize = sizeof(CREATEFILE2_EXTENDED_PARAMETERS); cf2ex.dwFileAttributes = FILE_ATTRIBUTE_NORMAL; cf2ex.hTemplateFile = NULL; h32 = CreateFile2(filePath->Data(), desiredAccess, shareMode, createDisposition, &cf2ex); if (h32 == INVALID_HANDLE_VALUE) { return; } flags &= _O_APPEND | _O_RDONLY | _O_TEXT; // only attributes described in http://msdn.microsoft.com/en-us/library/bdts1c9x.aspx hCrt = _open_osfhandle((intptr_t)h32, flags); nbytes = _write(hCrt, s_1kb_data, strlen(s_1kb_data)); if (nbytes == -1) { _close(hCrt); return; } if( _close(hCrt) == -1 ) { return; } uint64_t startTime = GetTickCount64(); // 2. get attribute by _fstat64() 10000 times for(int i=0 ; i<WRITE_FILE_COUNT ; i++) { // open file CREATEFILE2_EXTENDED_PARAMETERS cf2exStat = {0}; cf2exStat.dwSize = sizeof(CREATEFILE2_EXTENDED_PARAMETERS); cf2exStat.dwFileAttributes = FILE_ATTRIBUTE_NORMAL; cf2exStat.dwFileFlags = FILE_FLAG_BACKUP_SEMANTICS; hStat = CreateFile2(filePath->Data(), 0, FILE_SHARE_READ, OPEN_EXISTING, &cf2exStat); if (hStat == INVALID_HANDLE_VALUE) { return; } // open crt file handle int fd = _open_osfhandle((intptr_t)hStat, 0); if (fd == -1) { return; } // get file attribute struct __stat64 filestat; int rc = _fstat64(fd, &filestat); if (rc == -1) { _close(fd); return; } // close file _close(fd); } uint64_t endTime = GetTickCount64(); // (endTime - startTime) is the result } void RunTestCaseWinRT() { // 1. create file DWORD desiredAccess = GENERIC_READ|GENERIC_WRITE; DWORD shareMode = FILE_SHARE_DELETE | FILE_SHARE_READ | FILE_SHARE_WRITE; DWORD createDisposition = CREATE_ALWAYS;; HANDLE h32 = INVALID_HANDLE_VALUE; HANDLE hStat = INVALID_HANDLE_VALUE; int hCrt = 0; int flags = 0; int nbytes = 0; String^ filePath = ApplicationData::Current->LocalFolder->Path + "\\TestCaseWinRT.txt"; CREATEFILE2_EXTENDED_PARAMETERS cf2ex = {0}; cf2ex.dwSize = sizeof(CREATEFILE2_EXTENDED_PARAMETERS); cf2ex.dwFileAttributes = FILE_ATTRIBUTE_NORMAL; cf2ex.hTemplateFile = NULL; h32 = CreateFile2(filePath->Data(), desiredAccess, shareMode, createDisposition, &cf2ex); if (h32 == INVALID_HANDLE_VALUE) { return; } flags &= _O_APPEND | _O_RDONLY | _O_TEXT; // only attributes described in http://msdn.microsoft.com/en-us/library/bdts1c9x.aspx hCrt = _open_osfhandle((intptr_t)h32, flags); if (hCrt == -1) { return; } nbytes = _write(hCrt, s_1kb_data, strlen(s_1kb_data)); if (nbytes == -1) { _close(hCrt); return; } if( _close(hCrt) == -1 ) { return; } s_case7_loop_count = 0; s_case7_start_time = GetTickCount64(); // 2. open file create_task(ApplicationData::Current->LocalFolder->CreateFileAsync("TestCaseWinRT.txt",CreationCollisionOption::OpenIfExists)) .then( [this](StorageFile^ file) { // 3. start get attribute loop GetAttributeLoop(file); } ).then( [this](task<void> preTask) { try { preTask.get(); } catch(Exception^ ex) { ; } } ); } void GetAttributeLoop(Windows::Storage::StorageFile^ file) { //4. GetBasicPropertiesAsync create_task(file->GetBasicPropertiesAsync()) .then( [this,file](BasicProperties^ props) { s_case7_loop_count++; // 5. get until 10000 times if(s_case7_loop_count < WRITE_FILE_COUNT) { GetAttributeLoop(file); } else { //6. get attribute finished s_case7_end_time = GetTickCount64(); // (s_case7_end_time-s_case7_start_time) is the result } } ).then( [this](task<void> preTask) { try { preTask.get(); } catch(Exception^ ex) { ; } } ); }

     

    Both 2 test case try to get a file attributes with 1kb size it just create for 10000 times.

    CRT API takes 187 milliseconds.

    StorageFile API takes 11141 milliseconds.

    My questions are:

    1. Is there any behavior I did in my sample slow down the performance.

    2. If #1 is NO, is there any way to enhance the performance of StorageFile::GetBasicPropertiesAsync()? Or it is an expect performance of StorageFile API?

    Thursday, April 11, 2013 6:12 AM

All replies

  • My guess is that you're seeing thread switching/synchronization overhead in the Async case, not actual overhead in its retrieval of the file properties (which probably boils down to essentially the same Win32 APIs as fstat() internally).

    MSFT needs to move away from Async and back to regular threads ... but add better interthread communication.

    Thursday, April 11, 2013 11:22 PM
  • for WinRT APIs PPL always runs the continuation on the UI thread, so your code is going back and forth between threads and that adds overhead.

    in your create_task() call you can specify task_continuation_context::use_arbitrary() (the last optional param to create_task). that should change your results.

    Thursday, April 11, 2013 11:49 PM
  • Hi Chris,

    I modified my code of create_task() call for StorageFile::GetBasicPropertiesAsync() as following:

    void GetAttributeLoop(Windows::Storage::StorageFile^ file)
    {
        //4. GetBasicPropertiesAsync
        create_task(file->GetBasicPropertiesAsync())
         .then(
            [this,file](BasicProperties^ props)
            {
                s_case7_loop_count++;
                // 5. get until 10000 times
                if(s_case7_loop_count < WRITE_FILE_COUNT) {
                    GetAttributeLoop(file);
                }
                else {
                    //6. get attribute finished
                    s_case7_end_time = GetTickCount64();
                    // (s_case7_end_time-s_case7_start_time) is the result
                }
            }
        ,task_continuation_context::use_arbitrary()).then(
            [this](task<void> preTask)
            {
                try
                {
                    preTask.get();
                }
                catch(Exception^ ex)
                {
                    ;
                }
            }
        ,task_continuation_context::use_arbitrary());
    }

    But result seems to be the same as task_continuation_context::use_default (11140 milliseconds for 10000 times GetBasicPropertiesAsync()).

    Friday, April 12, 2013 3:11 AM
  •    Does anybody have other thoughts on this problem. It's really bothersome. I had to call GetBasicPropertiesAsync from C# and most likely I have different environment but my test results are terrible too. In my measures retrieving the file size using GetBasicPropertiesAsync takes about 52ms. It is for a single file and it's completely unacceptable when I have only 2000 files to request the size. We definitely need a faster way to do it but how?

    Alex

    Monday, September 15, 2014 3:49 PM