strange character encoding of stored data , old script is showing them fine new one doesn't

Question

I\'m trying to rewrite an old website .
it\'s in persian which uses perso/arabic characters .

CREATE DATABASE `db         


        

Answer 1

The deceze's answer is excellent but I can add some info that may help to handle lots of records without test them manually.

If the conversion CONVERT(BINARY CONVERT(field_name USING latin1) USING utf8) fails, it print NULL instead of the field_name content.

So I use this one to find those record:

SELECT IFNULL(
    CONVERT(BINARY CONVERT(field_name USING latin1) USING utf8)
    , '**************************************************')
FROM table_name

or this one:

SELECT id, field_name, CONVERT(BINARY CONVERT(field_name USING latin1) USING utf8)
FROM table_name
WHERE CONVERT(BINARY CONVERT(field_name USING latin1) USING utf8) IS NULL

And the UPDATE with the clause to affect only records on which the conversion success:

UPDATE table_name
SET
field_name = CONVERT(BINARY CONVERT(field_name USING latin1) USING utf8mb4 )
WHERE
CONVERT(BINARY CONVERT(field_name USING latin1) USING utf8mb4) IS NOT NULL