How to solve UnicodeDecodeError in Python 3.6?

Question

I am switched from Python 2.7 to Python 3.6.

I have scripts that deal with some non-English content.

I usually run scripts via Cron and also in Terminal.

Answer 1

For a Python-only solution you will have to recreate your sys.stdout object:

import sys, codecs
sys.stdout = codecs.getwriter('utf-8')(sys.stdout.detach())

After this, a normal print("hello world") should be encoded to UTF-8 automatically.

But you should try to find out why your terminal is set to such a strange encoding (which Python just tries to adopt to). Maybe your operating system is configured wrong somehow.

EDIT: In my tests unsetting the env variable LANG produced this strange setting for the stdout encoding for me:

LANG= python3
import sys
sys.stdout.encoding

printed 'ANSI_X3.4-1968'.

So I guess you might want to set your LANG to something like en_US.UTF-8. Your terminal program doesn't seem to do this.