locked
Building a JS app with grunt and UglifyJS - UTF-8 BOM problem

    Question

  • Hey,

    I'm currently working on porting my webapp, which is also available on other platforms, to Windows (8). As a webapp, it's mainly written in JavaScript, so I created a JS Windows Store App project and VS and moved in my app. Besides smaller bugs at the beginning, it's working flawlessly now, but I still have one bug I just can't resolve.

    I'm using grunt to streamline the build process. It uses grunt-contrib-uglify to concat and minify all my JS into small packages and grunt-contrib-cssmin to do the same to the CSS. All source files are correctly encoded as UTF-8 with BOM, I also did set the defaultEncoding for grunt to UTF-8 and enabled the preserveBOM option.

    Though, the minified files are encoded as codepage 1252. My app is running flawlessly with these files, but after packaging it for the Windows Store, certification fails. If I re-save all the packed files with the correct encoding, certification succeeds, but my app is not working anymore (screen stays white, seems like the JS file is not working anymore).

    Do you have tips for that scenario? I can't believe I'm the only one using such a setup.

    Thanks

    Wednesday, March 12, 2014 2:37 PM

All replies

  • Why is certification failing?

    Matt Small - Microsoft Escalation Engineer - Forum Moderator
    If my reply answers your question, please mark this post as answered.

    NOTE: If I ask for code, please provide something that I can drop directly into a project and run (including XAML), or an actual application project. I'm trying to help a lot of people, so I don't have time to figure out weird snippets with undefined objects and unknown namespaces.

    Wednesday, March 12, 2014 7:47 PM
    Moderator
  • The certification fails because my JS files are not correctly encoded as UTF-8 with BOM.

    I currently got the encoding fixed by finally adding the BOM (U+FEFF) to the beginnig of each of those files. All of them are now correctly UTF-8 encoded and certification succeeds. They just don't get loaded anymore.

    There is no error occuring, they just don't get run.

    I also tried to use the little endian version of the BOM (FFFE), ending with JS files, that are being loaded, but can't be interpreted and also certification fails.

    It's definitely not a bug in the scripts.

    Thursday, March 13, 2014 1:55 PM
  • This is what I'm doing with those JS files after concat/minification:

    var buf = grunt.file.read(fileName, { encoding: null });
    var missingBOM = (buf[0] !== 0xEF && buf[1] !== 0xBE && buf[2] !== 0xBB);
    if (missingBOM) {
        grunt.file.write(fileName, '\ufeff' + buf, { encoding: 'utf-8' });
    }


    • Edited by Ice8lue Thursday, March 13, 2014 4:00 PM
    Thursday, March 13, 2014 3:59 PM
  • I discovered that there are multiple BOMs in my merged files so I refined my script to strip those out before writing the new one:

    var buf = grunt.file.read(dist + fileName, { encoding: null });
    var BOM = new Buffer([0xEF,0xBB,0xBF]);
    
    // remove multi BOMs from Buffer
    var bufString = buf.toString('utf-8');
    bufString = bufString.replace(BOM.toString('utf-8'), null);
    buf = new Buffer(bufString, 'utf-8');
    
    // add new UTF-8 BOM to the beginning of the file buffer
    var bomFile = Buffer.concat([BOM,buf]);
    grunt.file.write(dist + fileName, bomFile, { encoding: 'utf-8' });

    Still, no luck -.-

    Friday, March 14, 2014 10:16 AM
  • So I found a solution now:

    The whole problem really seems to be connected to the uglify-merged files combined with the UTF8 BOM.  My deployment script now generates a LOT of single minified JS files instead of bundles and adds the corresponding BOM using the snippet from my last post. This solves the issue, but is ugly as hell...

    Tuesday, March 18, 2014 8:16 AM