Chinese and Japanese character support in python

Question

How to read correctly japanese and chinese characters. I\'m using python 2.5. Output is displayed as \"E:\\Test\\?????????\"

path = r\"E:\\Test\         


        

Answer 1

There are two independent issues:

1. You should specify Python source encoding if you use non-ascii characters and use Unicode literals for data that represents text e.g.:

   # -*- coding: utf-8 -*-
   path = ur"E:\Test\は最高のプログラマ"
   

2. Printing Unicode to Windows console is complicated but if you set correct font then just:

   print path
   

   might work.

Regardless of whether your console can display the path; it should be fine to pass the Unicode path to filesystem functions e.g.:

entries = os.listdir(path)

Don't call .encode(char_enc) on bytestrings, call it on Unicode strings instead.
Don't call .decode(char_enc) on Unicode strings, call it on bytestrings instead.