Warning raised by inserting 4-byte unicode to mysql

Question

Look at the following:

/home/kinka/workspace/py/tutorial/tutorial/pipelines.py:33: Warning: Incorrect string 
value: \'\\xF0\\x9F\\x91\\x8A\\xF0\\x9F...\' fo         


        

Answer 1

simple normalization for string without regex and translate:

def normalize_unicode(s):
    return ''.join([ unichr(k) if k < 0x10000 else 0xfffd for k in [ord(c) for c in s]])