I have a string "{'datetime': datetime.datetime(2010, 11, 21, 0, 56, 58)}" and I want to convert this to the object it represents. Using ast.literal_eval() gives
ValueError: malformed string; because it does not allow for construction of objects (i.e. the datetime call). Is there anyway to either get ast to process this correctly, or to secure eval to prevent code injection?
Following up on Ignacio Vazquez-Abrams' idea:
import ast import datetime def parse_datetime_dict(astr,debug=False): try: tree=ast.parse(astr) except SyntaxError: raise ValueError(astr) for node in ast.walk(tree): if isinstance(node,(ast.Module,ast.Expr,ast.Dict,ast.Str, ast.Attribute,ast.Num,ast.Name,ast.Load, ast.Tuple)): continue if (isinstance(node,ast.Call) and isinstance(node.func, ast.Attribute) and node.func.attr == 'datetime'): continue if debug: attrs=[attr for attr in dir(node) if not attr.startswith('__')] print(node) for attrname in attrs: print(' {k} ==> {v}'.format(k=attrname,v=getattr(node,attrname))) raise ValueError(astr) return eval(astr) good_strings=["{'the_datetime': datetime.datetime(2010, 11, 21, 0, 56, 58)}"] bad_strings=["__import__('os'); os.unlink", "import os; os.unlink", "import(os)", # SyntaxError ] for astr in good_strings: result=parse_datetime_dict(astr) print('{s} ... [PASSED]'.format(s=astr)) for astr in bad_strings: try: result=parse_datetime_dict(astr) except ValueError: print('{s} ... [REJECTED]'.format(s=astr)) else: sys.exit('ERROR: failed to catch {s!r}'.format(s=astr)) yields
{'the_datetime': datetime.datetime(2010, 11, 21, 0, 56, 58)} ... [PASSED] __import__('os'); os.unlink ... [REJECTED] import os; os.unlink ... [REJECTED] import(os) ... [REJECTED] You could extract the (2010, 11, 21, 0, 56, 58) characters from the string using a regex, pass that to ast.literal_eval() to get a tuple, and then pass that tuple to datetime.datetime(*a_tuple) to get the object. Sounds like a lot, but each of the steps is very simple (and secure). Here's what I'm talking about:
import ast import datetime import re s = "{'datetime': datetime.datetime(2010, 11, 21, 0, 56, 58)}" m = re.search(r"""datetime(\((\d+)(,\s*\d+)*\))""", s) if m: # any matches? args = ast.literal_eval(m.group(1)) print datetime.datetime(*args) # 2010-11-21 00:56:58 This searches for the pattern "datetime(, in the string and passes just the list of literal integer values to ast.literal_eval() for conversion to a tuple -- which should always succeed and is code-injection resistant. I believe it's called "Context-Sensitive String Evaluation" or CSSE.
Use the language services to compile it into an AST, walk the AST making sure that it contains only whitelisted node sets, then execute it.
Instead of writing lots of code, do not use ast when you have to parse datetime objs. You can run eval(). BTW please note that you may have security issues using this function if the string can contain dodgy python commands.
Here is how it works:
>>> x="{'datetime': datetime.datetime(2010, 11, 21, 0, 56, 58)}" >>> b=eval(x) >>> b {'datetime': datetime.datetime(2010, 11, 21, 0, 56, 58)} >>> b["datetime"].year 2010 Enjoy! :D
Use repr(datetime.datetime.utcnow()) to save in your dict or your file. Look,
>>> import datetime >>> oarg = datetime.datetime.utcnow() >>> oarg datetime.datetime(2013, 2, 6, 12, 39, 51, 709024) >>> butiwantthis = repr(oarg) >>> butiwantthis 'datetime.datetime(2013, 2, 6, 12, 39, 51, 709024)' >>> eval(oarg) Traceback (most recent call last): File "", line 1, in TypeError: eval() arg 1 must be a string, bytes or code object >>> eval(butiwantthis) datetime.datetime(2013, 2, 6, 12, 39, 51, 709024) Take care with import
>>> from datetime import datetime, date, time >>> oarg = datetime.datetime.utcnow() Traceback (most recent call last): File "", line 1, in AttributeError: type object 'datetime.datetime' has no attribute 'datetime' >>> oarg = datetime.utcnow() >>> oarg datetime.datetime(2013, 2, 6, 12, 41, 51, 870458) >>> butiwantthis = repr(oarg) >>> butiwantthis 'datetime.datetime(2013, 2, 6, 12, 41, 51, 870458)' >>> eval(butiwantthis) Traceback (most recent call last): File "", line 1, in File "", line 1, in AttributeError: type object 'datetime.datetime' has no attribute 'datetime' >>> # wrong import Perfect!
Ps: Python3.3