PHP/MySQL encoding problems. � instead of certain characters

Question

I have come across some problems when inputting certain characters into my mysql database using php. What I am doing is submitting user inputted text to a database. I cannot

Answer 1

The likeliest cause of the problem is that the database connection is set to latin1 but you are feeding it text encoded in UTF-8. The simplest way to solve this is to convert your input into what the client expects:

$quote = iconv("UTF-8", "WINDOWS-1252//TRANSLIT", $quote);

(What MySQL calls latin1 is windows-1252 in the rest of the world.) Note that many characters, such as the quotation dash U+2015 that you use there, cannot be represented in this encoding and will be converted into something else. Ideally you should change the column encoding to utf8.

An alternative solution: set the database connection to utf8. It doesn't matter how the columns are encoded: MySQL internally converts text from the connection encoding into the storage encoding, you can keep the columns as latin1 if you want to. (If you do, the quotation dash U+2015 will be turned into a question mark ? because it's not in latin1)

How to set the connection encoding depends on what library you are using: if you use the deprecated MySQL library it's mysql_set_charset, if MySQLi it's mysqli_set_charset, if PDO add encoding=utf8 to the DSN.

If you do this you'll have set the page encoding to UTF-8 with the Content-Type header. Otherwise you would be having the same problem with the browser: feeding it text encoded in UTF-8 when it's expecting something else:

header("Content-Type: text/html; charset=utf-8");