Linux/Python: encoding a unicode string for print

Question

I have a fairly large python 2.6 application with lots of print statements sprinkled about. I\'m using unicode strings throughout, and it usually works great. However, if I

Answer 1

Either wrap all your print statement through a method perform arbitrary unicode -> utf8 conversion or as last resort change the Python default encoding from ascii to utf-8 inside your site.py. In general it is a bad idea printing unicode strings unfiltered to sys.stdout since Python will trigger an implict conversion of unicode strings to the configured default encoding which is ascii.

Answer 2

I have now solved this problem. The solution was neither of the answers given. I used the method given at http://wiki.python.org/moin/PrintFails , as given by ChrisJ in one of the comments. That is, I replace sys.stdout with a wrapper that calls unicode encode with the correct arguments. Works very well.

Answer 3

If you're dumping to an ASCII terminal, encode manually using unicode.encode, and specify that errors should be ignored.

u = u'\xa0'
u.encode('ascii') # This fails
u.encode('ascii', 'ignore') # This replaces failed encoding attempts with empty string

If you want to store unicode files, try this:

u = u'\xa0'
print >>open('out', 'w'), u # This fails
print >>open('out', 'w'), u.encode('utf-8') # This is ok