GNU diff doesn\'t seem to be smart enough to detect and handle UTF-16 files, which surprises me. Am I missing an obvious command-line option? Is there a good alternative?<
You could maybe build something in python with the excellent chardet, then convert your files to UTF-8 and send this to GNU diff ?
http://chardet.feedparser.org/