locked
APFS Unicode Normalisation RRS feed

  • Question

  • User280114 posted

    Hi,

    With iOS 10.3, device file systems are being converted from HFS+ to APFS. HFS+ did Unicode normalisation at the file system level, but APFS doesn't and just stores whatever you give it as is. This is fine when working with either all iOS file access APIs or all .Net ones, but I'm having trouble when mixing them. As far as I can tell, by default, the .Net File APIs don't do any normalisation, but when I give a file path created & read with these APIs to NSUrl, it appears to be doing some normalisation, which creates a URL that doesn't exist because the actual file name was not normalised.

    This can be 'fixed' by manually normalising the file name before creating it with the .Net APIs, but updating an existing app to do this seems a bit cumbersome. I haven't yet found a way to create an NSUrl that doesn't mangle the name by doing unnecessary normalisation though.

    Can any Xamarin comment on whether they'll update the .Net APIs to handle this? Has any one found a nicer workaround yet?

    Thanks!

    Thursday, April 13, 2017 4:04 PM

All replies

  • User280114 posted

    bump?

    Tuesday, April 18, 2017 9:41 AM
  • User77604 posted

    I'm having the same problem. Did you find a solution to creating a NSUrl without normalisation?

    Thursday, May 4, 2017 6:16 PM
  • User64354 posted

    Unfortunately not. I ended up having to normalise all my file names when they were written instead. I could of done this with the .Net string methods but I thought it was safer to pass it through NSUrl just in case it used the non-standard normalisation form D like HFS+ did. I've put an example below, but stripped any error checking etc to keep it short.

    var fileUrl = Foundation.NSUrl.FromFilename(filePath);
    IntPtr fsRepPtr = fileUrl.GetFileSystemRepresentationAsUtf8Ptr;
    
    // We have to count the size manually as the methods for getting strings are
    // for UTF-16 or ASCII (not UTF-8) and the ASCII one returns the wrong size
    // when there are multi-byte characters (understandably).
    int numBytes = 0;
    while (System.Runtime.InteropServices.Marshal.ReadByte(fsRepPtr, numBytes) != 0)
    {
        numBytes++;
    }
    
    var utf8Bytes = new byte[numBytes];
    System.Runtime.InteropServices.Marshal.Copy(fsRepPtr, utf8Bytes, 0, numBytes);
    filePath = System.Text.Encoding.UTF8.GetString(utf8Bytes);
    
    Friday, May 5, 2017 8:17 AM
  • User64354 posted

    I appear to have ended up with two user accounts!

    Friday, May 5, 2017 8:19 AM
  • User200982 posted

    I got recently the same problem. My solution was to use String.Normalize on my string object.

    Friday, July 28, 2017 9:13 AM
  • User245475 posted

    I ended up writing an extension method for strings and calling that .ToNormalizedString() much cleaner than calling a function and you can pass params that determine how to do the encoding :D

    Friday, July 28, 2017 11:16 AM