What is the purpose of python's inner classes?

Question

Python's inner/nested classes confuse me. Is there something that can't be accomplished without them? If so, what is that thing?

Answer 1

Quoted from http://www.geekinterview.com/question_details/64739:

Advantages of inner class:

• Logical grouping of classes: If a class is useful to only one other class then it is logical to embed it in that class and keep the two together. Nesting such "helper classes" makes their package more streamlined.
• Increased encapsulation: Consider two top-level classes A and B where B needs access to members of A that would otherwise be declared private. By hiding class B within class A A's members can be declared private and B can access them. In addition B itself can be hidden from the outside world.
• More readable, maintainable code: Nesting small classes within top-level classes places the code closer to where it is used.

The main advantage is organization. Anything that can be accomplished with inner classes can be accomplished without them.

Answer 2

Is there something that can't be accomplished without them?

No. They are absolutely equivalent to defining the class normally at top level, and then copying a reference to it into the outer class.

I don't think there's any special reason nested classes are ‘allowed’, other than it makes no particular sense to explicitly ‘disallow’ them either.

If you're looking for a class that exists within the lifecycle of the outer/owner object, and always has a reference to an instance of the outer class — inner classes as Java does it – then Python's nested classes are not that thing. But you can hack up something like that thing:

import weakref, new

class innerclass(object):
    """Descriptor for making inner classes.

    Adds a property 'owner' to the inner class, pointing to the outer
    owner instance.
    """

    # Use a weakref dict to memoise previous results so that
    # instance.Inner() always returns the same inner classobj.
    #
    def __init__(self, inner):
        self.inner= inner
        self.instances= weakref.WeakKeyDictionary()

    # Not thread-safe - consider adding a lock.
    #
    def __get__(self, instance, _):
        if instance is None:
            return self.inner
        if instance not in self.instances:
            self.instances[instance]= new.classobj(
                self.inner.__name__, (self.inner,), {'owner': instance}
            )
        return self.instances[instance]


# Using an inner class
#
class Outer(object):
    @innerclass
    class Inner(object):
        def __repr__(self):
            return '<%s.%s inner object of %r>' % (
                self.owner.__class__.__name__,
                self.__class__.__name__,
                self.owner
            )

>>> o1= Outer()
>>> o2= Outer()
>>> i1= o1.Inner()
>>> i1
<Outer.Inner inner object of <__main__.Outer object at 0x7fb2cd62de90>>
>>> isinstance(i1, Outer.Inner)
True
>>> isinstance(i1, o1.Inner)
True
>>> isinstance(i1, o2.Inner)
False

(This uses class decorators, which are new in Python 2.6 and 3.0. Otherwise you'd have to say “Inner= innerclass(Inner)” after the class definition.)

Answer 3

There's something you need to wrap your head around to be able to understand this. In most languages, class definitions are directives to the compiler. That is, the class is created before the program is ever run. In python, all statements are executable. That means that this statement:

class foo(object):
    pass

is a statement that is executed at runtime just like this one:

x = y + z

This means that not only can you create classes within other classes, you can create classes anywhere you want to. Consider this code:

def foo():
    class bar(object):
        ...
    z = bar()

Thus, the idea of an "inner class" isn't really a language construct; it's a programmer construct. Guido has a very good summary of how this came about here. But essentially, the basic idea is this simplifies the language's grammar.

Answer 4

Nesting classes within classes:

• Nested classes bloat the class definition making it harder to see whats going on.

• Nested classes can create coupling that would make testing more difficult.

• In Python you can put more than one class in a file/module, unlike Java, so the class still remains close to top level class and could even have the class name prefixed with an "_" to help signify that others shouldn't be using it.

The place where nested classes can prove useful is within functions

def some_func(a, b, c):
   class SomeClass(a):
      def some_method(self):
         return b
   SomeClass.__doc__ = c
   return SomeClass

The class captures the values from the function allowing you to dynamically create a class like template metaprogramming in C++

Answer 5

I understand the arguments against nested classes, but there is a case for using them in some occasions. Imagine I'm creating a doubly-linked list class, and I need to create a node class for maintaing the nodes. I have two choices, create Node class inside the DoublyLinkedList class, or create the Node class outside the DoublyLinkedList class. I prefer the first choice in this case, because the Node class is only meaningful inside the DoublyLinkedList class. While there's no hiding/encapsulation benefit, there is a grouping benefit of being able to say the Node class is part of the DoublyLinkedList class.

Answer 6

I have used Python's inner classes to create deliberately buggy subclasses within unittest functions (i.e. inside def test_something():) in order to get closer to 100% test coverage (e.g. testing very rarely triggered logging statements by overriding some methods).

In retrospect it's similar to Ed's answer https://stackoverflow.com/a/722036/1101109

Such inner classes should go out of scope and be ready for garbage collection once all references to them have been removed. For instance, take the following inner.py file:

class A(object):
    pass

def scope():
    class Buggy(A):
        """Do tests or something"""
    assert isinstance(Buggy(), A)

I get the following curious results under OSX Python 2.7.6:

>>> from inner import A, scope
>>> A.__subclasses__()
[]
>>> scope()
>>> A.__subclasses__()
[<class 'inner.Buggy'>]
>>> del A, scope
>>> from inner import A
>>> A.__subclasses__()
[<class 'inner.Buggy'>]
>>> del A
>>> import gc
>>> gc.collect()
0
>>> gc.collect()  # Yes I needed to call the gc twice, seems reproducible
3
>>> from inner import A
>>> A.__subclasses__()
[]

Hint - Don't go on and try doing this with Django models, which seemed to keep other (cached?) references to my buggy classes.

So in general, I wouldn't recommend using inner classes for this kind of purpose unless you really do value that 100% test coverage and can't use other methods. Though I think it's nice to be aware that if you use the __subclasses__(), that it can sometimes get polluted by inner classes. Either way if you followed this far, I think we're pretty deep into Python at this point, private dunderscores and all.

Answer 7

The main use case I use this for is the prevent proliferation of small modules and to prevent namespace pollution when separate modules are not needed. If I am extending an existing class, but that existing class must reference another subclass that should always be coupled to it. For example, I may have a utils.py module that has many helper classes in it, that aren't necessarily coupled together, but I want to reinforce coupling for some of those helper classes. For example, when I implement https://stackoverflow.com/a/8274307/2718295

:utils.py:

import json, decimal

class Helper1(object):
    pass

class Helper2(object):
    pass

# Here is the notorious JSONEncoder extension to serialize Decimals to JSON floats
class DecimalJSONEncoder(json.JSONEncoder):

    class _repr_decimal(float): # Because float.__repr__ cannot be monkey patched
        def __init__(self, obj):
            self._obj = obj
        def __repr__(self):
            return '{:f}'.format(self._obj)

    def default(self, obj): # override JSONEncoder.default
        if isinstance(obj, decimal.Decimal):
            return self._repr_decimal(obj)
        # else
        super(self.__class__, self).default(obj)
        # could also have inherited from object and used return json.JSONEncoder.default(self, obj) 

Then we can:

>>> from utils import DecimalJSONEncoder
>>> import json, decimal
>>> json.dumps({'key1': decimal.Decimal('1.12345678901234'), 
... 'key2':'strKey2Value'}, cls=DecimalJSONEncoder)
{"key2": "key2_value", "key_1": 1.12345678901234}

Of course, we could have eschewed inheriting json.JSONEnocder altogether and just override default():

:

import decimal, json

class Helper1(object):
    pass

def json_encoder_decimal(obj):
    class _repr_decimal(float):
        ...

    if isinstance(obj, decimal.Decimal):
        return _repr_decimal(obj)

    return json.JSONEncoder(obj)


>>> json.dumps({'key1': decimal.Decimal('1.12345678901234')}, default=json_decimal_encoder)
'{"key1": 1.12345678901234}'

But sometimes just for convention, you want utils to be composed of classes for extensibility.

Here's another use-case: I want a factory for mutables in my OuterClass without having to invoke copy:

class OuterClass(object):

    class DTemplate(dict):
        def __init__(self):
            self.update({'key1': [1,2,3],
                'key2': {'subkey': [4,5,6]})


    def __init__(self):
        self.outerclass_dict = {
            'outerkey1': self.DTemplate(),
            'outerkey2': self.DTemplate()}



obj = OuterClass()
obj.outerclass_dict['outerkey1']['key2']['subkey'].append(4)
assert obj.outerclass_dict['outerkey2']['key2']['subkey'] == [4,5,6]

I prefer this pattern over the @staticmethod decorator you would otherwise use for a factory function.