python-unicode

问题 I had been trying to animate bengali characters using Manim. I used this method to use pc fonts in Manim. Everything seemed to be working well until i saw the output. For instance, if i write বাংলা লেখা i get the output as (look closely at the output) বাংলা লখো. Most of the times it spits out absolutely meaningless words. The code used was: class test_3(Scene): def construct(self): text1 = Text('বাংলা লেখা', font='Akaash') text2 = Text('english text', font='Arial').move_to(DOWN) self.play

问题 I'm working with dbf database and Armenian letters, the DBF encoding was unknown so I've created a letter map to decode revived string. Now I have a valid Unicode string, but I cannot print it out because of this error: UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-5: character maps to What I have tried so far: print u'%s' %str ## Returns mentioned error print repr(str) ## Returns string in this form u'\u054c\u0561\u0586\u0561\u0575\u0565\u056c How to fix it? 回答1:

问题 I've downloaded some web pages with requests and saved the content in a postgres database [in a text field] using Django's ORM. For some sudocode of what's going on, here ya go: art = Article() page = requests.get("http://example.com") art.raw_html = page.content art.save() I verified that page.content is a bytes object, and I guess I assumed that this object would automatically be decoded upon saving, but it doesn't seem to be... it has been converted to some weird string representation of a

问题 I've downloaded some web pages with requests and saved the content in a postgres database [in a text field] using Django's ORM. For some sudocode of what's going on, here ya go: art = Article() page = requests.get("http://example.com") art.raw_html = page.content art.save() I verified that page.content is a bytes object, and I guess I assumed that this object would automatically be decoded upon saving, but it doesn't seem to be... it has been converted to some weird string representation of a

问题 I have a file containing UTF-16 strings. When I try to read the unicode, " " (double quotes) are added and the string looks like "b'\\xff\\xfeA\\x00'" . The inbuilt .decode function throws a AttributeError: 'str' object has no attribute 'decode' . I tried a few options but those didn't work. This is what the file I am reading from looks like 回答1: Try this: str.encode().decode() 回答2: It looks like the file has been created by writing bytes literals to it, something like this: some_bytes = b

问题 I have a file containing UTF-16 strings. When I try to read the unicode, " " (double quotes) are added and the string looks like "b'\\xff\\xfeA\\x00'" . The inbuilt .decode function throws a AttributeError: 'str' object has no attribute 'decode' . I tried a few options but those didn't work. This is what the file I am reading from looks like 回答1: Try this: str.encode().decode() 回答2: It looks like the file has been created by writing bytes literals to it, something like this: some_bytes = b

问题 I have the following dataset 'Fʀɪᴇɴᴅ', 'ᴍᴏᴍ', 'ᴍᴀᴋᴇs', 'ʜᴏᴜʀʟʏ', 'ᴛʜᴇ', 'ᴄᴏᴍᴘᴜᴛᴇʀ', 'ʙᴇᴇɴ', 'ᴏᴜᴛ', 'ᴀ', 'ᴊᴏʙ', 'ғᴏʀ', 'ᴍᴏɴᴛʜs', 'ʙᴜᴛ', 'ʟᴀsᴛ', 'ᴍᴏɴᴛʜ', 'ʜᴇʀ', 'ᴄʜᴇᴄᴋ', 'ᴊᴜsᴛ', 'ᴡᴏʀᴋɪɴɢ', 'ғᴇᴡ', 'ʜᴏᴜʀs', 'sᴏᴜʀᴄᴇ', I want then into ASCII format using Python script for example: Fʀɪᴇɴᴅ - FRIEND ᴍᴏᴍ - MOM I have tried encoding decoding but that doesn't work i also have tried this solution. but that doesn't solve my problem. 回答1: Python doesn't provide a way to directly convert small caps

问题 I am working on an NLP task that requires using a corpus of the language called Yoruba. Yoruba is a language that has diacritics (accents) and under dots in its alphabets. For instance, this is a Yoruba string: "ọmọàbúròẹlẹ́wà" , and I need to remove the accents and keep the under dots. I have tried using the unidecode library in Python, but it removes accents and under dots. import unidecode ac_stng = "ọmọàbúròẹlẹ́wà" unac_stng = unidecode.unidecode(ac_stng) I expect the output to be

问题 I have a foreign C library which uses utf-16 in API: as function arguments, return values and structure members. On Windows its OK with ctypes.c_wchar_p, but under OSX ctypes uses UCS-32 in c_wchar and I could not find the way to support utf-16. Here is my research: Use _SimpleCData subclassing to redefine _check_retval_. it allows a transparent conversion of utf-16 to Python string. can be placed as C structure member But it doesn't allow to handle strings as arguments, its from_param()

问题 I have a foreign C library which uses utf-16 in API: as function arguments, return values and structure members. On Windows its OK with ctypes.c_wchar_p, but under OSX ctypes uses UCS-32 in c_wchar and I could not find the way to support utf-16. Here is my research: Use _SimpleCData subclassing to redefine _check_retval_. it allows a transparent conversion of utf-16 to Python string. can be placed as C structure member But it doesn't allow to handle strings as arguments, its from_param()