Overload Python 'in' to return non-bool

Question

I\'m trying to overload the in operator for a class to return a non-bool object, but it seems to cast anyway. Here is my use case:

class Dataset         


        

Answer 1

There are two problems here. First, in always casts the result of __contains__ to a bool, so what you're looking for isn't possible. The second problem is that

fruit in ['apple', 'orange']

calls

['apple', 'orange'].__contains__(fruit)

There's no way for the left operand of in to override the operator, so that's also going to defeat what you're trying to do.

Answer 2

The real problem here is that, as the documentation says:

For user-defined classes which define the __contains__() method, x in y is true if and only if y.__contains__(x) is true.

Also see here and here.

user2357112's answer explains this better than I could. You're calling list.__contains__, not FilterBuilder.__contains__.

---

But why does it work that way?

Well, how else could it work?

Imagine that 3 in [1, 2, 3] called int.__contains__. And so did 3 in {1, 2, 3}, and 3 in my_custom_sorted_bintree. How could int.__contains__(container) possibly be implemented? Certainly not by iterating over the container. That would mean slow, exhaustive search for looking things up in sets and bintrees, which would defeat the whole point. My bintree class might not even be iterable, and yet may still have a concept of membership.

But what if they call list.__contains__, set.__contains__ and CustomSortedBintree.__contains__? Don't they have to know about int, and str, and every other possible thing you could give them? No, not at all. A list just has to know how to compare arg == elem for each of its elements. A set also needs to know how to call hash(arg). A bintree also has to know how to call arg < elem. But you don't need to know anything about the type of arg to do that.

---

You probably want to know how to deal with this. There are two and a half common solutions.

1: You can easily create a FilterList class. Then, you just write:

fruit in FilterList('apple', 'orange')

1.5: Or, with a bit more work, you can build a more general "value holder":

fruit in const(['apple', 'orange'])

2: Alternatively, you can write a FilterBuilder.in_ method. Then you write:

fruit.in_(['apple', 'orange'])

… or, if you prefer:

fruit.in_('apple', 'orange')

Most libraries I've seen either provide the second (sqlalchemy), or provide both but use the second in their tutorials (appscript), although "quick-lambda" libraries often go with the generic version of the first.

But you should consider the tradeoffs for your own use case. In general, the first is easier to implement, and more explicit, and has the advantage that subfilters/subqueries can return something that acts as a FilterList; the second is less verbose and arguably easier to read.

If neither one is acceptable, you could consider writing a parser for a Python-like DSL, instead of trying to build a DSL out of actual Python code via expression templates. Or using something like MacroPy (which I think even has an example similar to what you're looking for—as well as quick-lambda macros that don't need "const" and friends).

Answer 3

No. list.__contains__ always returns a bool, and C types cannot be monkeypatched (nor should you consider doing so, since you will likely break other code).