Fix encoding using command line or msbuild

Question

We compile many small related js files into one using MSBuild. And have some issue with encoding:

File encoding

FAILED

UTF-8 file encoding

• Error Found: The UTF-8 file encoding test detected the following errors:
  • File C:\Program Files\WindowsApps\someapp\somepath\compiled.js is not properly UTF-8 encoded. Re-save the file as UTF-8 (including Byte Order Mark).
• Impact if not fixed: HTML, CSS, and JavaScript files must be encoded in UTF-8 form with a corresponding byte-order mark (BOM) in order to benefit from bytecode caching and to avoid other runtime error conditions.
• How to fix: Open the affected file, and select "Save As..." option from the File menu in Visual Studio. Select the drop-down control next to the Save button and select "Save with Encoding”... option. From the Advanced save options dialog, choose the "Unicode (UTF-8 with signature)" option and click the OK button. 

Can we fix it automatically using msbuild or something command line tools? Not by clicking buttons in visual studio.

Answer 1

Hello,

Changing file's encoding using command line. I am not really sure but you can try

recode UTF-8 [filename]

I also found some other method for batch convert files encoding. please refer:

http://superuser.com/questions/16672/how-can-i-convert-multiple-files-to-utf-8-encoding-using-nix-command-line-tools

http://stackoverflow.com/questions/2311750/change-file-encoding-to-utf-8-via-vim-in-a-script

http://mindspill.net/computing/linux-notes/determine-and-change-file-character-encoding/

Hope this helps.thanks.

---

Yanping Wang
MSDN Community Support | Feedback to us
Develop and promote your apps in Windows Store
Please remember to mark the replies as answers if they help and unmark them if they provide no help.

Answer 2

Oh. Its all about *nix. I do not have these problems in the *nix.

Answer 3

Find workaround whats work in my case. Just put BOM to top of file before compile it.

  <ItemGroup>
    <CompileOutput Include="js\player\compiled.js" />
    <CompileInput Include="js\player\**\*.js" Exclude="@(CompiledPlugins)" />
  </ItemGroup>

  <Target Name="CompilePlugins"
          Inputs="@(CompileInput)"
          Outputs="@(CompileOutput)">

    <PropertyGroup>
      <BOMCode>65279</BOMCode>
      <BOM>$([System.Convert]::ToChar( $([System.Int32]::Parse( $(BOMCode) )) ))</BOM>
    </PropertyGroup>

    <!-- clean old file and write BOM into it  -->
    <WriteLinesToFile File="@(CompileOutput)"
                      Overwrite="true"
                      Lines="$(BOM)" />

    <!-- Calls ones for each file so cannot use override here -->
    <WriteLinesToFile File="@(CompileOutput)"
                      Overwrite="false"
                      Lines="$([System.IO.File]::ReadAllText(%(CompileInput.Identity)))" />

Answer 4

Anton,

There is a much simpler way:

  <ItemGroup>
      <CacheFiles Include="$(ProjectDir)\**\*.js" />
    </ItemGroup>
    <WriteLinesToFile File="%(CacheFiles.FullPath)" Lines="$([System.IO.File]::ReadAllText(%(CacheFiles.FullPath)))" Overwrite="true" Encoding="utf-8" />  

By using the Encoding attribute, you will get the BOM header.

Todd