sandboxing/running python code line by line

Question

I\'d love to be able to do something like these two are doing:

Inventing on principle @18:20 , Live ClojureScript Game Editor

If you don\'t wanna check the v

Answer 1

Okay guys, I've made a bit progress.

Say we have a source file like this, we want to run statement by statement:

print("single line")
for i in xrange(3):
    print(i)
    print("BUG, executed outside for-scope, so only run once")
if i < 0:
    print("Should not get in here")
if i > 0:
    print("Should get in here though")

I want to execute it one statement at a time, while having access to the locals/globals. This is a quick dirty proof of concept (disregard the bugs and crudeness):

# returns matched text if found
def re_match(regex, text):
    m = regex.match(text)
    if m: return m.groups()[0]

# regex patterns
newline = "\n"
indent = "[ ]{4}"
line = "[\w \"\'().,=<>-]*[^:]"
block = "%s:%s%s%s" % (line, newline, indent, line)

indent_re = re.compile(r"^%s(%s)$" % (indent, line))
block_re = re.compile(r"^(%s)$" % block)
line_re =  re.compile(r"^(%s)$" % (line))

buf = ""
indent = False

# parse the source using the regex-patterns
for l in source.split(newline):
    buf += l + newline              # add the newline we removed by splitting

    m = re_match(indent_re, buf)    # is the line indented?
    if m: 
        indent = True               # yes it is
    else:
        if indent:                  # else, were we indented previously?
            indent = False          # okay, now we aren't

    m = re_match(block_re, buf)     # are we starting a block ?
    if m:
        indent = True
        exec(m)
        buf = ""
    else:
        if indent: buf = buf[4:]   # hack to remove indentation before exec'ing
        m = re_match(line_re, buf) # single line statement then?
        if m:
            exec(m) # execute the buffer, reset it and start parsing
            buf = ""
        # else no match! add a line more to the buffer and try again

Output:

morten@laptop /tmp $ python p.py
single line
0
1
2
BUG, executed outside for-scope, son only run once
Should get in here though

So this is somewhat what I want. This code breaks the source into executable statements and I'm able to "pause" in between statements and manipulate the environment. As the code above shows, I can't figure out how to properly break up the code and execute it again. This made me think that I should be able to use some tool to parse the code and run it like I want. Right now I'm thinking ast or pdb like you guys suggest.

A quick look suggests ast can do this, but it seems a bit complex so I'll have to dig into the docs. If pdb can control the flow programmatically, that may very well be the answer too.

Update:

Sooo, I did some more reading and I found this topic: What cool hacks can be done using sys.settrace?

I looked into using sys.settrace(), but it doesn't seem to be the way to go. I am getting more and more convinced I need to use the ast module to get as fine-gained control as I would like to. FWIW here's the code to use settrace() to peak inside function scope vars:

import sys

def trace_func(frame,event,arg):
    print "trace locals:"
    for l in frame.f_locals:
        print "\t%s = %s" % (l, frame.f_locals[l])

def dummy(ls):
    for l in ls: pass

sys.settrace(trace_func)
x = 5
dummy([1, 2, 3])
print "whatisthisidonteven-"

output:

morten@laptop /tmp $ python t.py 
trace locals:
    ls = [1, 2, 3]
whatisthisidonteven-
trace locals:
    item = 
    selfref = 
trace locals:
    item = 
    selfref = 

UPDATE:

Okay I seem to have solved it.. :) Ive written a simple parser that injects a statement between each line of code and then executes the code.. This statement is a function call that captures and saves the local environment in its current state.

I'm working on a Tkinter text editor with two windows that'll do what Bret Victor does in his binarySearch-demo. I'm almost done :)