Grunt and UglifyJS in Windows app development - UTF8 problems

问题

I'm developing a Windows 8/8.1 JavaScript app. It basicallly is a port of our webapp. For deployment, I'm using grunt, grunt-contrib-uglify and grunt-contrib-cssmin (next to some more, but those are not connected to my problem.

UglifyJS is used to package all my JS files and libs in a minified bundle for each. For the CSS it's the same using CSSmin.

As the Windows app certification forces a UTF-8 encoding with signature (BOM), all my source files are encoded that way. I also did set the grunt default encoding to UTF-8 with preserveBOM to true. Though, my minified files ar encoded in us_ascii, some even in binary stream.

That behavior is the same for Windows 8.1 32bit and Mac OS Mavericks.

What can I do?

Re-saving this packed version with UTF-8 BOM encoding leads to a passed certification check, but the app is not running (white screen, looks like the minified JS is not working anymore, but it does flawlessly before re-saving).

This is what I'm doing with the JS files after concat/minify:

var buf = grunt.file.read(fileName, { encoding: null });
var missingBOM = (buf[0] !== 0xEF && buf[1] !== 0xBE && buf[2] !== 0xBB);
if (missingBOM) {
  grunt.file.write(fileName, '\ufeff' + buf, { encoding: 'utf-8' });
}

Update: I discovered that there are multiple BOMs in my merged files so I refined my script to strip those out before writing the new one:

var buf = grunt.file.read(dist + fileName, { encoding: null });
var BOM = new Buffer([0xEF,0xBB,0xBF]);

// remove multi BOMs from Buffer
var bufString = buf.toString('utf-8');
bufString = bufString.replace(BOM.toString('utf-8'), null);
buf = new Buffer(bufString, 'utf-8');

// add new UTF-8 BOM to the beginning of the file buffer
var bomFile = Buffer.concat([BOM,buf]);
grunt.file.write(dist + fileName, bomFile, { encoding: 'utf-8' });

Still, no luck -.-

回答1:

I had a similar problem, though a lot of circumstances were diff't, so not sure if solution will work for you.

Uglify unescapes and converts \u0000 and other unicode chars in regexps to utf-8.

Try executing uglify with the options ascii-only=true and beautify=false

more discussion at this uglify issue

回答2:

My guess is that it is your Windows loading with the wrong charset. Did you set the charset in your html page so that it would force the browser to load it using UTF-8?

It seems to be a problem with how your files is being loaded instead of how they are generated/minified.

Link: http://www.w3.org/International/questions/qa-html-encoding-declarations

<html>
 <head>
  <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
 </head>
 <body>
 </body>
</html>

回答3:

The Problem really seems to be the combination of uglify merged files and BOM. My current setup minifies each JS file but doesn't merge them anymore (there's no need to, as the Windows app will store all scripts locally so requests are not a problem). I'm adding the BOM to all of them at the end using the snippet from update 2 of my main post.

My app now successfully certifies and runs flawlessly. Phew...

来源：https://stackoverflow.com/questions/22352909/grunt-and-uglifyjs-in-windows-app-development-utf8-problems

标签

javascript

node.js

encoding

utf-8

gruntjs