How to determine if a String contains invalid encoded characters

Question

Usage scenario

We have implemented a webservice that our web frontend developers use (via a php api) internally to display product data. On the webs

Answer 1

You need to setup the character encoding from the start. Try sending the proper Content-Type header, for example Content-Type: text/html; charset=utf-8 to fix the right encoding. The standard conformance refers to utf-8 and utf-16 as the proper encoding for Web Services. Examine your response headers.

Also, at the server side — in the case which the browser do not handles properly the encoding sent by the server — force the encoding by allocating a new String. Also you can check each byte in the encoded utf-8 string by doing a single each_byte & 0x80, verifying the result as non zero.

boolean utfEncoded = true;
byte[] strBytes = queryString.getBytes();
for (int i = 0; i < strBytes.length(); i++) {
    if ((strBytes[i] & 0x80) != 0) {
        continue;
    } else {
        /* treat the string as non utf encoded */
        utfEncoded = false;
        break;
    }
}

String realQueryString = utfEncoded ?
    queryString : new String(queryString.getBytes(), "iso-8859-1");

Also, take a look on this article, I hope it would help you.