__classcell__ generates error in Python 3.6 when the metaclass calls multiple super().__new__ from inherited class

送分小仙女□ 提交于 2021-02-10 14:39:16

问题


Here is an executable code which works in Python 2.7 but results in an error in Python 3.6:

import six

class AMeta(type):

    def __new__(cls, name, bases, attrs):
        module = attrs.pop('__module__')
        new_attrs = {'__module__': module}
        classcell = attrs.pop('__classcell__', None)
        if classcell is not None:
            new_attrs['__classcell__'] = classcell
        new = super(AMeta, cls).__new__(
            cls, name, bases, new_attrs)
        new.duplicate = False
        legacy = super(AMeta, cls).__new__(
            cls, 'Legacy' + name, (new,), new_attrs)
        legacy.duplicate = True
        return new

@six.add_metaclass(AMeta)
class A():
    def pr(cls):
        print('a')

class B():
    def pr(cls):
        print('b')

class C(A,B):
    def pr(cls):
        super(C, cls).pr() # not shown with new_attrs
        B.pr(cls)
        print('c') # not shown with new_attrs
c = C()
c.pr()

# Expected result
# a
# b
# c

I get the following error:

Traceback (most recent call last):
  File "test.py", line 28, in <module>
    class C(A,B):
TypeError: __class__ set to <class '__main__.LegacyC'> defining 'C' as <class '__main__.C'>

C is inherit from A that is generated with the metaclass AMeta. They are tests classes and AMeta's goal is to execute all the tests with 2 different file folders: the default one and the legacy one.

I found a way to remove thise error by removing classcell from attrs, then returning new = super(AMeta, cls).new(cls, name, bases, attrs) (not new_attrs) but it doesn't seem right, and if it is, I'd like to know why.

The goal of new_attrs resulted from this SO question or from the documentation where it states basically the opposite: when modifying the attrs, make sure to keep classcell because it is deprecated in Python 3.6 and will result in an error in Python 3.8. Note that in this case, it removes the pr definition because they weren't passed to new_attrs, thus prints 'b' instead of 'abc', but is irrelevant for this problem.

Is there a way to call multiple super().new inside the new of a metaclass AMeta, and then call them from a class C inheriting from the class inheriting A ?

Without nesting inheritance, the error doesn't appear, like this:

import six

class AMeta(type):

    def __new__(cls, name, bases, attrs):
        new = super(AMeta, cls).__new__(
            cls, name, bases, attrs)
        new.duplicate = False
        legacy = super(AMeta, cls).__new__(
            cls, 'Duplicate' + name, (new,), attrs)
        legacy.duplicate = True
        return new

@six.add_metaclass(AMeta)
class A():
    def pr(cls):
        print('a')

a = A()
a.pr()

# Result

# a

Then maybe it is A's role to do something to fix it?

Thanks by advance,


回答1:


What your problem is I can figure out, and how to work around it The problem is that when you do what you are doing, you are passing the same cell object to both copies of your class: the original and the legacy one.

As it exists in two classes at once, it conflicts with the other place it is in use when one tries to make use of it - super() will pick the wrong ancestor class when called.

cell objects are picky, they are created in native code, and can't be created or configured on the Python side. I could figure out a way of creating the class copy by having a method that will return a fresh cell object, and passing that as __classcell__.

(I also tried to simply run copy.copy/copy.deepcopy on the classcell object -before resorting to my cellfactory bellow - it does not work)

In order to reproduce the problem and figure out a solution I made a simpler version of your metaclass, Python3 only.

from types import FunctionType
legacies = []

def classcellfactory():
    class M1(type):
        def __new__(mcls, name, bases, attrs, classcellcontainer=None):
            if isinstance(classcellcontainer, list):
                classcellcontainer.append(attrs.get("__classcell__", None))

    container = []

    class T1(metaclass=M1, classcellcontainer=container):
        def __init__(self):
            super().__init__()
    return container[0]


def cellfactory():
    x = None
    def helper():
        nonlocal x
    return helper.__closure__[0]

class M(type):
    def __new__(mcls, name, bases, attrs):
        cls1 = super().__new__(mcls, name + "1", bases, attrs)
        new_attrs = attrs.copy()
        if "__classcell__" in new_attrs:
            new_attrs["__classcell__"] = cellclass = cellfactory()

            for name, obj in new_attrs.items():
                if isinstance(obj, FunctionType) and obj.__closure__:
                    new_method = FunctionType(obj.__code__, obj.__globals__, obj.__name__, obj.__defaults__, (cellclass, ))
                    new_attrs[name] = new_method

        cls2 = super().__new__(mcls, name + "2", bases, new_attrs) 
        legacies.append(cls2)
        return cls1

class A(metaclass=M):
    def meth(self):
        print("at A")

class B(A): 
    pass

class C(B,A): 
    def meth(self):
        super().meth()

C()

So, not only I create a nested-function in order to have the Python runtime create a separate cell object, that I then use in the cloned class - but also, methods that make use of the cellclass have to be re-created with a new __closure__ that points to the new cell var.

Without recreating the methods, they won't work in the clonned class - as super() in the cloned-class' methods will expect the cell pointing to the cloned class itself, but it points to the original one.

Fortunately, methods in Python 3 are plain functions - that makes the code simpler. However, that code won't run in Python 2 - so, just enclose it in an if block not to run on Python2. As the __cellclass__ attribute does not even exist there, there is no problem at all.

After pasting the code above in a Python shell I can run both methods and super() works:

In [142]: C().meth()                                                                                                                              
at A

In [143]: legacies[-1]().meth()                                                                                                                   
at A


来源:https://stackoverflow.com/questions/55614454/classcell-generates-error-in-python-3-6-when-the-metaclass-calls-multiple-su

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!