django unicode encode/decode errors

Question

My site needs to be able to serve data in different languages. I set it so it uses utf-8 and the db settings are set to that as well. I've been getting different different unicode errors over the admin.

For example:

1. In the admin list, when a field from the list contains a non ascii char. (i get UnicodeDecodeError)
2. When adding a new entry, a UnicodeEncodeError if the unicode method for the model returns an utf-8 decode (which fixes #1).
   
3. When using a filter_horizontal in the admin, if data from the used model contains non ascii chars, then the filter disappears from the form.
   

If I set the unicode method for the model to return for example:

return u'%s' % unicode(self.tag)

That seems to fix #1 and #2, but then that's when I get #3.

I have been looking a lot for a solution, but can't find something that fixes all different errors. What's the best way to deal with those?

Answer 1

from django.utils.encoding import smart_unicode
...
def __unicode__(self): 
    return smart_unicode(self.tag)

Answer 2

It is noteworthy that you can bypass unicode by simply encoding your data in hexadecimal before storing it in your database.

Something like this is sufficient

MyModel(name=name.encode('hex'), password=password).save()

You can then execute name.decode('hex') to return the data back to its former representation.