Avoiding long constructors while inheriting without hiding constructor, optional arguments or functionality

Question

I have a particular problem, but I will make the example more general. I have a Parent class with a mandatory constructor parameter and a few optional ones,

Answer 1

The basic idea is to write code that generates the __init__ method for you, with all the parameters specified explicitly rather than via *args and/or **kwargs, and without even needing to repeat yourself with all those self.arg1 = arg1 lines.

And, ideally, it can make it easy to add type annotations that PyCharm can use for popup hints and/or static type checking.¹

And, while you're at it, why not build a __repr__ that displays the same values? And maybe even an __eq__, and a __hash__, and maybe lexicographical comparison operators, and conversion to and from a dict whose keys match the attributes for each JSON persistence, and…

Or, even better, use a library that takes care of that for you.

Python 3.7 comes with such a library, dataclasses. Or you can use a third-party library like attrs, that works with Python 3.4 and (with some limitations) 2.7. Or, for simple cases (where your objects are immutable, and you want them to work like a tuple of their attributes in specified order), you can use namedtuple, which works back to 3.0 and 2.6.

Unfortunately, dataclasses doesn't quite work for your use case. If you just write this:

from dataclasses import dataclass

@dataclass
class Parent:
    arg1: str
    opt_arg1: str = 'opt_arg1_default_val'
    opt_arg2: str = 'opt_arg2_default_val'
    opt_arg3: str = 'opt_arg3_default_val'
    opt_arg4: str = 'opt_arg4_default_val'

@dataclass
class Child(Parent):
    arg2: str

… you'll get an error, because it tries to place the mandatory parameter arg2 after the default-values parameters opt_arg1 through opt_arg4.

dataclasses doesn't have any way to reorder parameters (Child(arg1, arg2, opt_arg1=…), or to force them to be keyword-only parameters (Child(*, arg1, opt_arg1=…, arg2)). attrs doesn't have that functionality out of the box, but you can add it.

So, it's not quite as trivial as you'd hope, but it's doable.

---

But if you wanted to write this yourself, how would you create the __init__ function dynamically?

The simplest option is exec.

You've probably heard that exec is dangerous. But it's only dangerous if you're passing in values that came from your user. Here, you're only passing in values that came from your own source code.

It's still ugly—but sometimes it's the best answer anyway. The standard library's namedtuple used to be one giant exec template., and even the current version uses exec for most of the methods, and so does dataclasses.

Also, notice that all of these modules store the set of fields somewhere in a private class attribute, so subclasses can easily read the parent class's fields. If you didn't do that, you could use the inspect module to get the Signature for your base class's (or base classes', for multiple inheritance) initializer and work it out from there. But just using base._fields is obviously a lot simpler (and allows storing extra metadata that doesn't normally go in signatures).

Here's a dead simple implementation that doesn't handle most of the features of attrs or dataclasses, but does order all mandatory parameters before all optionals.

def makeinit(cls):
    fields = ()
    optfields = {}
    for base in cls.mro():
        fields = getattr(base, '_fields', ()) + fields
        optfields = {**getattr(base, '_optfields', {}), **optfields}
    optparams = [f"{name} = {val!r}" for name, val in optfields.items()]
    paramstr = ', '.join(['self', *fields, *optparams])
    assignstr = "\n    ".join(f"self.{name} = {name}" for name in [*fields, *optfields])
    exec(f'def __init__({paramstr}):\n    {assignstr}\ncls.__init__ = __init__')
    return cls

@makeinit
class Parent:
    _fields = ('arg1',)
    _optfields = {'opt_arg1': 'opt_arg1_default_val',
                  'opt_arg2': 'opt_arg2_default_val',
                  'opt_arg3': 'opt_arg3_default_val',
                  'opt_arg4': 'opt_arg4_default_val'}

@makeinit
class Child(Parent):
    _fields = ('arg2',)

Now, you've got exactly the __init__ methods you wanted on Parent and Child, fully inspectable² (including help), and without having to repeat yourself.

---

_{1. I don't use PyCharm, but I know that well before 3.7 came out, their devs were involved in the discussion of @dataclass and were already working on adding explicit support for it to their IDE, so it doesn't even have to evaluate the class definition to get all that information. I don't know if it's available in the current version, but if not, I assume it will be. Meanwhile, @dataclass already just works for me with IPython auto-completion, emacs flycheck, and so on, which is good enough for me. :)}

_{2. … at least at runtime. PyCharm may not be able to figure things out statically well enough to do popup completion.}