efficiently replace bad characters

Question

I often work with utf-8 text containing characters like:

\\xc2\\x99

\\xc2\\x95

\\xc2\\x85

etc

<

Answer 1

import unicodedata

# Convert to unicode
text_to_uncicode = unicode(text, "utf-8")           

# Convert back to ascii
text_fixed = unicodedata.normalize('NFKD',text_to_unicode).encode('ascii','ignore')