PHP character encoding hell reading csv file with fgets

Question

I have a web site that receives a CSV file by FTP once a month. For years it was an ASCII file. Now I\'m receiving UTF-8 one month then UTF-16BE the next and UTF-16LE the m

Answer 1

Explicitly pass the order and possible encodings to detect, and use strict parameter. Also please use file_get_contents, if the file is in UTF-16LE, fgets will screw it up for you.

$encoding
";

foreach( explode( PHP_EOL, $input ) as $line ) {
    var_dump( $line );
}

The order is important because UTF-8 and UTF-32 are more restrictive and UTF-16 is extremely permissive; pretty much any random even length of bytes are valid UTF-16.

The only way you will retain all information, is to convert it to an unicode encoding, not ASCII.