Python script receiving a UnicodeEncodeError: 'ascii' codec can't encode character

Question

I have a simple Python script that pulls posts from reddit and posts them on Twitter. Unfortunately, tonight it began having issues that I\'m assuming are because of someone\'s

Answer 1

You are trying to print a unicode string to your terminal (or possibly a file by IO redirection), but the encoding used by your terminal (or file system) is ASCII. Because of this Python attempts to convert it from the unicode representation to ASCII, but fails because codepoint u'\u201c' (“) can not be represented in ASCII. Effectively your code is doing this:

>>> print u'\u201c'.encode('ascii')
Traceback (most recent call last):
  File "", line 1, in 
UnicodeEncodeError: 'ascii' codec can't encode character u'\u201c' in position 0: ordinal not in range(128)

You could try converting to UTF-8:

print (post + " " + post_dict[post] + " #python").encode('utf8')

or convert to ASCII like this:

print (post + " " + post_dict[post] + " #python").encode('ascii', 'replace')

which will replace invalid ASCII characters with ?.

Another way, which is useful if you are printing for debugging purposes, is to print the repr of the string:

print repr(post + " " + post_dict[post] + " #python")

which would output something like this:

>>> s = 'string with \u201cLEFT DOUBLE QUOTATION MARK\u201c'
>>> print repr(s)
u'string with \u201cLEFT DOUBLE QUOTATION MARK\u201c'