I'm trying to read in a string representation of a Tuple from a file, and add the tuple to a list. Here's the relevant code.
raw_data = userfile.read().split('\n') for a in raw_data : print a btc_history.append(ast.literal_eval(a))
Here is the output:
(Decimal('11.66985'), Decimal('0E-8')) Traceback (most recent call last): File "./goxnotify.py", line 74, in <module> main() File "./goxnotify.py", line 68, in main local.load_user_file(username,btc_history) File "/home/unix-dude/Code/GoxNotify/local_functions.py", line 53, in load_user_file btc_history.append(ast.literal_eval(a)) File "/usr/lib/python2.7/ast.py", line 80, in literal_eval return _convert(node_or_string) `File "/usr/lib/python2.7/ast.py", line 58, in _convert return tuple(map(_convert, node.elts)) File "/usr/lib/python2.7/ast.py", line 79, in _convert raise ValueError('malformed string') ValueError: malformed string
ast.literal_eval (located in ast.py) parses the tree with ast.parse first, then it evaluates the code with quite an ugly recursive function, interpreting the parse tree elements and replacing them with their literal equivalents. Unfortunately the code is not at all expandable, so to add Decimal to the code you need to copy all the code and start over.
For a slightly easier approach, you can use ast.parse module to parse the expression, and then the ast.NodeVisitor or ast.NodeTransformer to ensure that there is no unwanted syntax or unwanted variable accesses. Then compile with compile and eval to get the result.
The code is a bit different from literal_eval in that this code actually uses eval, but in my opinion is simpler to understand and one does not need to dig too deep into AST trees. It specifically only allows some syntax, explicitly forbidding for example lambdas, attribute accesses (foo.__dict__ is very evil), or accesses to any names that are not deemed safe. It parses your expression fine, and as an extra I also added Num (float and integer), list and dictionary literals.
Also, works the same on 2.7 and 3.3
import ast import decimal source = "(Decimal('11.66985'), Decimal('1e-8'),"\ "(1,), (1,2,3), 1.2, [1,2,3], {1:2})" tree = ast.parse(source, mode='eval') # using the NodeTransformer, you can also modify the nodes in the tree, # however in this example NodeVisitor could do as we are raising exceptions # only. class Transformer(ast.NodeTransformer): ALLOWED_NAMES = set(['Decimal', 'None', 'False', 'True']) ALLOWED_NODE_TYPES = set([ 'Expression', # a top node for an expression 'Tuple', # makes a tuple 'Call', # a function call (hint, Decimal()) 'Name', # an identifier... 'Load', # loads a value of a variable with given identifier 'Str', # a string literal 'Num', # allow numbers too 'List', # and list literals 'Dict', # and dicts... ]) def visit_Name(self, node): if not node.id in self.ALLOWED_NAMES: raise RuntimeError("Name access to %s is not allowed" % node.id) # traverse to child nodes return self.generic_visit(node) def generic_visit(self, node): nodetype = type(node).__name__ if nodetype not in self.ALLOWED_NODE_TYPES: raise RuntimeError("Invalid expression: %s not allowed" % nodetype) return ast.NodeTransformer.generic_visit(self, node) transformer = Transformer() # raises RuntimeError on invalid code transformer.visit(tree) # compile the ast into a code object clause = compile(tree, '<AST>', 'eval') # make the globals contain only the Decimal class, # and eval the compiled object result = eval(clause, dict(Decimal=decimal.Decimal)) print(result)
From the documentation for ast.literal_eval():
Safely evaluate an expression node or a string containing a Python expression. The string or node provided may only consist of the following Python literal structures: strings, numbers, tuples, lists, dicts, booleans, and None.
Decimal isn't on the list of things allowed by ast.literal_eval().
