subclassed python dictionary for custom namespace in exec() method

Question

I am trying to subclass a dictionary for use in an exec method. Ultimately, I would like the local function to have a custom name scoping behaviour.

In the code below, function b() does in fact have the correct globals() dictionary available to it, however it fails to search it when resolving z.

Does function b() first not search locals() then globals() ?

Very puzzling. Any help appreciated.

t = '''
def b():
#   return (globals()['z']) #works
    return z #fails

b()
'''

class MyDict(dict):
    def __init__(self, g):
        dict.__init__(self)
        self.my_g = g


    def __getitem__(self, key):
        print("GET ", key)
        try:
            val = dict.__getitem__(self, key)
        except:
            print("GET exception1")
            val = self.my_g[key]
        return val


g = {'z':123}

md = MyDict(g)

#fails to find z
exec(t, md, md)

#works
#exec(t, g, g)

output

GET  b
Traceback (most recent call last):
  File "/project1/text12", line 31, in <module>
  File "<string>", line 6, in <module>
  File "<string>", line 4, in b
NameError: global name 'z' is not defined

Answer 1

Before python 3.3, you cannot use a custom dict subclass for the globals value of an exec statement. The underlying C code that executes the compiled code accesses the underlying C structures directly, ignoring any custom hooks you may have implemented.

In other words, when the code does a LOAD_GLOBAL operation (as is the case with z in your function b), the C bytecode evaluation loop accesses the globals structure in the current frame using the C API, bypassing any python overrides.

This is documented in the exec() function documentation as:

If only globals is provided, it must be a dictionary, which will be used for both the global and the local variables. If globals and locals are given, they are used for the global and local variables, respectively. If provided, locals can be any mapping object.

This restriction has been loosened in Python 3.3, see issue 14385. The documentation hasn't been updated yet, but the bytecode evaluation loop has been updated to test for custom mappings before falling back to the C API access. If a custom mapping is used, the PyObject_GetItem function is used, which will call __getitem__ on custom classes.