Check for invalid UTF8

Question

I am converting from UTF8 format to actual value in hex. However there are some invalid sequences of bytes that I need to catch. Is there a quick way to check if a characte

Answer 1

public static bool IsValidUtf8(byte[] bytes, int length)
{
    // https://en.wikipedia.org/wiki/UTF-8#Codepage_layout
    // http://www.w3.org/International/questions/qa-forms-utf-8
    // https://social.msdn.microsoft.com/Forums/vstudio/en-US/df18cca9-5e54-410e-a5c5-74efc7b52e29
    // http://gallery.technet.microsoft.com/scriptcenter/ConvertTo-String-d79aed45

    Encoding enc = Encoding.GetEncoding("iso-8859-1");
    string binaryText = enc.GetString(bytes, 0, length);
    Regex rx = new Regex(@"\A(
          [\x09\x0A\x0D\x20-\x7E]            # ASCII
        | [\xC2-\xDF][\x80-\xBF]             # non-overlong 2-byte
        |  \xE0[\xA0-\xBF][\x80-\xBF]        # excluding overlongs
        | [\xE1-\xEC\xEE\xEF][\x80-\xBF]{2}  # straight 3-byte
        |  \xED[\x80-\x9F][\x80-\xBF]        # excluding surrogates
        |  \xF0[\x90-\xBF][\x80-\xBF]{2}     # planes 1-3
        | [\xF1-\xF3][\x80-\xBF]{3}          # planes 4-15
        |  \xF4[\x80-\x8F][\x80-\xBF]{2}     # plane 16
        )*\z", RegexOptions.IgnorePatternWhitespace | RegexOptions.Singleline);

    return rx.IsMatch(binaryText);
}

Note:

To be able to match regular expression against binary data (bytes), the binary data is converted first to a Unicode string (all .net strings are Unicode) using "iso-8859-1" encoding. It is the only single-byte encoding that has one-to-one mapping with the first 256 Unicode code points. Other encodings do not preserve all the binary bytes after conversion to text.

static void Main(string[] args)
{
    string filename = "myfile.txt";
    byte[] bytes = File.ReadAllBytes(filename);

    if (IsValidUtf8(bytes, bytes.Length))
    {
        Console.WriteLine("encoding: utf-8");
    }        
    else
    {
        Console.WriteLine("unknown encoding.");
    }
}