Reading an UTF-8 encoded text file in Mathematica

Question

How can I read a utf-8 encoded text file in Mathematica?

This is what I\'m doing now:

text = Import[\"charData.txt\", \"Text\", Char         


        

Answer 1

Short version: Mathematica's UTF-8 functionality does not work for character codes with more than 16 bits. Use UTF-16 encoding instead, if possible. But be aware that Mathematica's treatment of 17+ bit character codes is generally buggy. The long version follows...

As noted by numerous commenters, the problem appears to be with Mathematica's support for Unicode characters whose codes are larger than 16 bits. The first such character in the cited text file is U+20B9B (