Detecting 'text' file type (ANSI vs UTF-8)

Question

I wrote an application (a psychological testing exam) in Delphi (7) which creates a standard text file - ie the file is of type ANSI.

Someone has ported the program

Answer 1

When reading first try parsing the file as UTF-8. If it isn't valid UTF-8 interpret the file as the legacy encoding(ANSI). This will work on most files, since it's very unlikely that a legacy encoded file will be valid UTF-8.

What windows calls ANSI is a system locale dependent charset. And the text won't work correctly on a Russian, or Asian or... windows.

While the VCL doesn't support Unicode in Delphi 7, you still should internally work with unicode and only convert to ANSI for displaying it. I localized one of my programs to Korean and Russian, and that was the only way I got it working without large problems. You still could only display the Korean localization on a system set to Korean, but at least the text-files could be edited on any system.