ast's col_offset is different between local and App Engine?

问题

My project on App Engine Python3.7 environment tried using ast module:

from flask import Flask, request
import ast

app = Flask(__name__)

@app.route('/', methods=['GET'])
def respond():
    code = '''
if True:
    pass
elif True:
    pass
elif False:
    pass
else:
    pass
'''
    module_node = ast.parse(code)
    visitor = MyVisitor()
    visitor.visit(module_node)
    return visitor.result

class MyVisitor(ast.NodeVisitor):
    def visit_If(self, node):
        self.result = '{} {} {} {}'.format(
            node.col_offset,
            node.orelse[0].col_offset,
            node.orelse[0].orelse[0].col_offset,
            node.orelse[0].orelse[0].orelse[0].col_offset
        )

if __name__ == '__main__':
    app.run(host='127.0.0.1', port=8080, debug=True)

The result was '0 5 5 4' on local (Python3.7.4 on macOS 10.15.4). However App Engine returned '0 0 0 4'. How can I resolve this? I need ast to parse code and generate another representation. Thank you.

回答1:

Looks like this changed between Python 3.7.6 and Python 3.7.7:

import ast
import sys


class MyVisitor(ast.NodeVisitor):
    def visit_If(self, node):
        self.result = '{} {} {} {}'.format(
            node.col_offset,
            node.orelse[0].col_offset,
            node.orelse[0].orelse[0].col_offset,
            node.orelse[0].orelse[0].orelse[0].col_offset
        )


code = '''
if True:
    pass
elif True:
    pass
elif False:
    pass
else:
    pass
'''
module_node = ast.parse(code)
visitor = MyVisitor()
visitor.visit(module_node)
print(sys.version)
print(visitor.result)

$ python3.7.6 test.py
3.7.6 (default, May 20 2020, 09:51:40)
[Clang 11.0.0 (clang-1100.0.33.16)]
0 5 5 4

$ python3.7.7 test.py
3.7.7 (default, Apr 24 2020, 10:25:06)
[Clang 11.0.0 (clang-1100.0.33.16)]
0 0 0 4

At the time of writing, App Engine's python37 runtime uses Python 3.7.7 and is generally upgraded to the latest patch version as it is released.

I would expect the behavior of the newer version of Python to be more "correct" here, and in fact this behavior persists through 3.8 and 3.9 as well:

$ python3.8 test.py
3.8.2 (default, Apr 22 2020, 21:21:01)
[Clang 11.0.0 (clang-1100.0.33.16)]
0 0 0 4

$ python3.9 test.py
3.9.0a5 (default, Apr 24 2020, 10:32:31)
[Clang 11.0.0 (clang-1100.0.33.16)]
0 0 0 4

来源：https://stackoverflow.com/questions/61904643/asts-col-offset-is-different-between-local-and-app-engine

标签

python

google-app-engine