how do I .decode('string-escape') in Python3?

问题:

I have some escaped strings that need to be unescaped. I'd like to do this in Python.

For example, in python2.7 I can do this:

>>> "\123omething special".decode('string-escape') 'Something special' >>>  

How do I do it in Python3? This doesn't work:

>>> b"\123omething special".decode('string-escape') Traceback (most recent call last):   File "", line 1, in  LookupError: unknown encoding: string-escape >>>  

My goal is to be abel to take a string like this:

s\000u\000p\000p\000o\000r\000t\000@\000p\000s\000i\000l\000o\000c\000.\000c\000o\000m\000 

And turn it into:

"support@psiloc.com" 

After I do the conversion, I'll probe to see if the string I have is encoded in UTF-8 or UTF-16.

回答1:

You'll have to use unicode_escape instead:

>>> b"\\123omething special".decode('unicode_escape') 

If you start with a str object instead (equivalent to the python 2.7 unicode) you'll need to encode to bytes first, then decode with unicode_escape.

If you need bytes as end result, you'll have to encode again to a suitable encoding (.encode('latin1') for example, if you need to preserve literal byte values; the first 255 unicode code points map 1-on-1).

Your example is actually UTF-16 data with escapes. Decode from unicode_escape, back to latin1 to preserve the bytes, then from utf-16-le (UTF 16 little endian without BOM):

>>> value = b's\\000u\\000p\\000p\\000o\\000r\\000t\\000@\\000p\\000s\\000i\\000l\\000o\\000c\\000.\\000c\\000o\\000m\\000' >>> value.decode('unicode_escape').encode('latin1')  # convert to bytes b's\x00u\x00p\x00p\x00o\x00r\x00t\x00@\x00p\x00s\x00i\x00l\x00o\x00c\x00.\x00c\x00o\x00m\x00' >>> _.decode('utf-16-le') # decode from UTF-16-LE 'support@psiloc.com' 

回答2:

The old "string-escape" codec maps bytestrings to bytestrings, and there's been a lot of debate about what to do with such codecs, so it isn't currently available through the standard encode/decode interfaces.

BUT, the code is still there in the C-API (as PyBytes_En/DecodeEscape), and this is still exposed to Python via the undocumented codecs.escape_encode and codecs.escape_decode.

>>> import codecs >>> codecs.escape_decode(b"ab\\xff") (b'ab\xff', 6) >>> codecs.escape_encode(b"ab\xff") (b'ab\\xff', 3) 

These functions return the transformed bytes object, plus a number indicating how many bytes were processed... you can just ignore the latter.

>>> value = b's\\000u\\000p\\000p\\000o\\000r\\000t\\000@\\000p\\000s\\000i\\000l\\000o\\000c\\000.\\000c\\000o\\000m\\000' >>> codecs.escape_decode(value)[0] b's\x00u\x00p\x00p\x00o\x00r\x00t\x00@\x00p\x00s\x00i\x00l\x00o\x00c\x00.\x00c\x00o\x00m\x00' 

回答3:

You can't use unicode_escape on byte strings (or rather, you can, but it doesn't always return the same thing as string_escape

This function implements string_escape using a regular expression and custom replacement logic.

def unescape(text):     regex = re.compile(b'\\\\(\\\\|[0-7]{1,3}|x.[0-9a-f]?|[\'"abfnrt]|.|$)')     def replace(m):         b = m.group(1)         if len(b) == 0:             raise ValueError("Invalid character escape: '\\'.")         i = b[0]         if i == 120:             v = int(b[1:], 16)         elif 48 

文章来源: how do I .decode('string-escape') in Python3?