How do I treat an ASCII string as unicode and unescape the escaped characters in it in python?

Question

For example, if I have a unicode string, I can encode it as an ASCII string like so:

>>> u\'\\u003cfoo/\\u003e\'.encode(\'ascii\')         


        

Answer 1

At some point you will run into issues when you encounter special characters like Chinese characters or emoticons in a string you want to decode i.e. errors that look like this:

UnicodeEncodeError: 'ascii' codec can't encode characters in position 109-123: ordinal not in range(128)

For my case (twitter data processing), I decoded as follows to allow me to see all characters with no errors

>>> s = '\u003cfoo\u003e'
>>> s.decode( 'unicode-escape' ).encode( 'utf-8' )
>>>