Guessing the encoding of text represented as byte[] in Java

Question

Given an array of bytes representing text in some unknown encoding (usually UTF-8 or ISO-8859-1, but not necessarily so), what is the best way to obtain a guess for the most

Answer 1

Here's my favorite: https://github.com/codehaus/guessencoding

It works like this:

• If there's a UTF-8 or UTF-16 BOM, return that encoding.
• If none of the bytes have the high-order bit set, return ASCII (or you can force it to return a default 8-bit encoding instead).
• If there are bytes with the high bit set but they're arranged in the correct patterns for UTF-8, return UTF-8.
• Otherwise, return the platform default encoding (e.g., windows-1252 on an English-locale Windows system).

It may sound overly simplistic, but in my day-to-day work it's well over 90% accurate.