Functional pipes in python like %>% from R's magritrr

Question

In R (thanks to magritrr) you can now perform operations with a more functional piping syntax via %>%. This means that instead of coding this: <

Answer 1

Building pipe with Infix

As hinted at by Sylvain Leroux, we can use the Infix operator to construct a infix pipe. Let's see how this is accomplished.

First, here is the code from Tomer Filiba

^{Code sample and comments by Tomer Filiba (http://tomerfiliba.com/blog/Infix-Operators/) :}

from functools import partial

class Infix(object):
    def __init__(self, func):
        self.func = func
    def __or__(self, other):
        return self.func(other)
    def __ror__(self, other):
        return Infix(partial(self.func, other))
    def __call__(self, v1, v2):
        return self.func(v1, v2)

Using instances of this peculiar class, we can now use a new "syntax" for calling functions as infix operators:

>>> @Infix
... def add(x, y):
...     return x + y
...
>>> 5 |add| 6

The pipe operator passes the preceding object as an argument to the object that follows the pipe, so x %>% f can be transformed into f(x). Consequently, the pipe operator can be defined using Infix as follows:

In [1]: @Infix
   ...: def pipe(x, f):
   ...:     return f(x)
   ...:
   ...:

In [2]: from math import sqrt

In [3]: 12 |pipe| sqrt |pipe| str
Out[3]: '3.4641016151377544'

A note on partial application

The %>% operator from dpylr pushes arguments through the first argument in a function, so

df %>% 
filter(x >= 2) %>%
mutate(y = 2*x)

corresponds to

df1 <- filter(df, x >= 2)
df2 <- mutate(df1, y = 2*x)

The easiest way to achieve something similar in Python is to use currying. The toolz library provides a curry decorator function that makes constructing curried functions easy.

In [2]: from toolz import curry

In [3]: from datetime import datetime

In [4]: @curry
    def asDate(format, date_string):
        return datetime.strptime(date_string, format)
    ...:
    ...:

In [5]: "2014-01-01" |pipe| asDate("%Y-%m-%d")
Out[5]: datetime.datetime(2014, 1, 1, 0, 0)

Notice that |pipe| pushes the arguments into the last argument position, that is

x |pipe| f(2)

corresponds to

f(2, x)

When designing curried functions, static arguments (i.e. arguments that might be used for many examples) should be placed earlier in the parameter list.

Note that toolz includes many pre-curried functions, including various functions from the operator module.

In [11]: from toolz.curried import map

In [12]: from toolz.curried.operator import add

In [13]: range(5) |pipe| map(add(2)) |pipe| list
Out[13]: [2, 3, 4, 5, 6]

which roughly corresponds to the following in R

> library(dplyr)
> add2 <- function(x) {x + 2}
> 0:4 %>% sapply(add2)
[1] 2 3 4 5 6

Using other infix delimiters

You can change the symbols that surround the Infix invocation by overriding other Python operator methods. For example, switching __or__ and __ror__ to __mod__ and __rmod__ will change the | operator to the mod operator.

In [5]: 12 %pipe% sqrt %pipe% str
Out[5]: '3.4641016151377544'

Answer 2

There is very nice pipe module here https://pypi.org/project/pipe/ It overloads | operator and provide a lot of pipe-functions like add, first, where, tail etc.

>>> [1, 2, 3, 4] | where(lambda x: x % 2 == 0) | add
6

>>> sum([1, [2, 3], 4] | traverse)
10

Plus it's very easy to write own pipe-functions

@Pipe
def p_sqrt(x):
    return sqrt(x)

@Pipe
def p_pr(x):
    print(x)

9 | p_sqrt | p_pr