none
Windows CE 6.0 exFAT file system RRS feed

  • General discussion

  • Hello, I have the following doubt. Does the exFAT file system (in WinCE 6.0) support natively the file data caching ? I have read on MSDN documentation that it supports disk caching (the metadata and FAT caching) but for file data caching we need to use a file cache manager that is a file system filter put on top of exFAT. So if I use only exFAT and not include File Cache Manager catalog item into my image, I haven't support for file caching ? If I use the write file API, is the data written directly into mass storage ?

    Best regards, Paolo.

    Wednesday, October 20, 2010 10:57 AM

All replies

  • Even without file cache and disk cache, it doesn't imply the data will be written to storage directly.
    Unless you specific Write Through mode when open the file (CreateFile), some of the data may still in the file system or storage driver's internal write buffer.
    Wednesday, October 20, 2010 7:00 PM
  • So, as you said, I can bypass cache using FILE_FLAG_WRITE_THROUGH into dwFlagsAndAttributes parameter in CreateFile API ? Further, You said "storage driver's internal write buffer"...is it a buffer always present in a file system that has no relationship with file caching a disk caching ? If yes, what are the advantages to use file caching and disk caching added on internal write buffer ?

    Wednesday, October 20, 2010 7:48 PM
  • Unlike cache, the write buffer of File system or Storage driver just like it name, a  buffer between under layer IO and the software.
    Because the block device driver (Storage) or the File System storage (e.g. a FAT cluster) need to be accessed via a basic unit (sector/cluster); the purpose of write buffer is to have a temporally memory for this basic unit so upper layer software does not need to access data aligned to the basic unit size.

    When you using WriteFile API, you can write whatever size you want, so it is possible some of the data may not drain to the storage but still in the write buffer. With FILE_FLAG_WRITE_THROUGH flag, it indicates file system and block driver to drain the write buffer on each accessing.

    Thursday, October 21, 2010 1:09 AM
  • So if I have understood there are the following layer in top-down fashion :

    - User Application

    - File Caching Manager (as a filter for file system)

    - File System (with the following features always in top-down fashion)

    • Data Caching (that supports metadata caching and file contents caching);
    • Write Buffer

    - Storage Driver

    • WriteBuffer

    - Storage Device (HW)

    Is it right ?

    So with File Caching I can manage caching on whole file contents but with write buffer (cache) I can manage caching of a single basic unit (sector/cluster). So If I want a direct write into storage without any kind of caching, I have to :

    • disable file caching manager;
    • using FILE_FLAG_WRITE_THROUGH into CreateFile API;

    Is it right ?

    The data caching inside file system is usefull because I can avoid metadata lost (ex. FAT corruption) when the power gows down. Is it right ?

    Thanks, Paolo

    Thursday, October 21, 2010 6:10 AM
  • Your recognition is right. And with well designed filesystem and cache filter, when CreateFile with FILE_FLAG_WRITE_THROUGH flag, the cache should be ignored to that file automatically and drain the write buffer on each WriteFile.
    And if data integrity is your priority, you should consider to use TFAT. (FormatTfat http://msdn.microsoft.com/en-us/library/ee490783.aspx and other FAT relative topic)

     

    Friday, October 22, 2010 3:59 PM
  • Why did you say "the cache should be ignored" ? Why "should" ? Isn't it sure ? I already use TFAT and I have another question : to be sure that metadata file system information (ex. FAT) are written directly on mass storage, I have to disable disk caching ?

    Thanks for your patience.

    Best regards, Paolo.

    Friday, October 22, 2010 5:58 PM
  • With shared source program, we can only access source code of cache filter (private\winceos\coreos\fsd\cachefilt) but not FAT. So we can only exam the cache filter and trust the FAT.
    The DoOpenFile in private\winceos\coreos\fsd\cachefilt\CachedVolume.cpp, it indicates underlying FS not to cache by FILE_FLAG_NO_BUFFERING and FLAG_FLAG_WRITE_THROUGH flag, so as long as the FAT honors this flag (no source code so assume it is correct), CreateFile with FLAG_FLAG_WRITE_THROUGH should guarantee File Data will be flushed to disk immediately.
    Regard to the metadata, with TFAT enabled, it should guarantee the metadata is flushed on time to ensure data integrity (again no source code to verify) regardless using with Disk Cache or not.
    Sunday, October 24, 2010 6:51 PM
  • So if We use Cache filter and CreateFile API with FILE_FLAG_NO_BUFFERING and FLAG_FLAG_WRITE_THROUGH flags, we can suppose that FAT honors these flags and We have to suppose the same also if we don't use cache filter. One doubt : in CachedVolume.cpp I can found FILE_FLAG_NO_BUFFERING use but not FLAG_FLAG_WRITE_THROUGH.

    Only last question : the mass storage I have the TFAT is a NAND Flash. If I use FLAG_FLAG_WRITE_THROUGH to write file modifications immediately, does the NAND Flash life reduce ? In this way I increment the number of write operation beacuse I read and write a block even if I modify a single byte into the file...is it true ? With caching, more modification are cached and then all are flushed on NAND Flash with a single write operation.

    Thanks, Paolo.

    Monday, October 25, 2010 7:15 AM
  • Basically, you are right for the concern of wear leveling when accessing file in write through mode.
    If you want to analyze the wear leveling in your using scenario, you can use RAMFMD (public\common\oak\drivers\block\msflashfmd\ram) with TrackWearLeveling or port the idea to your driver so you can track the wear leveling info when accessing file with and without write through mode.
    Tuesday, October 26, 2010 5:34 AM
  • Thank you very much KMOS for your explanation.

    Paolo.

    Tuesday, October 26, 2010 3:56 PM