How to handle Unicode (non-ASCII) characters in Python?

Question

I\'m programming in Python and I\'m obtaining information from a web page through the urllib2 library. The problem is that that page can provide me with non-ASCII characters

Answer 1

You just read a set of bytes from the socket. If you want a string you have to decode it:

yourstring = receivedbytes.decode("utf-8") 

(substituting whatever encoding you're using for utf-8)

Then you have to do the reverse to send it back out:

outbytes = yourstring.encode("utf-8")