How do I diff utf-16 files with GNU diff?

Question

GNU diff doesn\'t seem to be smart enough to detect and handle UTF-16 files, which surprises me. Am I missing an obvious command-line option? Is there a good alternative?<

Answer 1

Install ripgrep utility which supports UTF-16, then run:

diff <(rg -N . file1.txt) <(rg -N . file2.txt)

ripgrep supports searching files in text encodings other than UTF-8, such as UTF-16, latin-1, GBK, EUC-JP, Shift_JIS and more. (Some support for automatically detecting UTF-16 is provided. Other text encodings must be specifically specified with the -E/--encoding flag.)