Transmit parameter to the inner-most call of a composed function

问题

I'm building a data pipelining tool and want to be able to compile a series of functions together.

All the functions act on an iterable, and pass an iterable as a yielded value.

So if f(x) filters, and g(x) edits, and h(x) generates a random value, then I want to be able to compose combinations of these so I can call f(g(h(x))) or h(f(g(x))) as per requirements.

I'd like to be able to prepare these compositions (they have different parameter signatures beyond the initial iterable) before knowing what x is going to be.

To complicate matters, f, g and h, while sharing the iterable parameter that they both consume and emit, they have different parameter signatures.

For convenience, I can wrap these parameters up in a partial statement to hide the messy parameters - but next I want to compose a series of functions as per the f(g(h(x))) example - the idea being that I want to shoe-horn a particular x iterable at runtime.

I'm finding that having partial'ed the functions, subsequently nesting them means I can't access that deepest parameter - and I'm getting errors like AttributeError: 'generator' object has no attribute 'keywords'.

In other words, is there a way to chain or nest functions in a way that allows you to defer specifying a parameter required by the innermost function at runtime?

e.g. The following works fine:

data_source = [ OrderedDict({"id" : "1", "name" : "Tom", "sync" : "a"}),
            OrderedDict({"id" : "2", "name" : "Steve", "sync" : "a"}),
            OrderedDict({"id" : "3", "name" : "Ulrich", "sync" : "b"}),
            OrderedDict({"id" : "4", "name" : "Victor", "sync" : "b"}),
            OrderedDict({"id" : "5", "name" : "Wolfgang", "sync" : "c"}),
            OrderedDict({"id" : "6", "name" : "Xavier", "sync" : "c"}),
            OrderedDict({"id" : "7", "name" : "Yves", "sync" : "c"}),
            OrderedDict({"id" : "8", "name" : "Zaphod", "sync" : "d"}),
           OrderedDict({"id" : "9", "name" : "Albert", "sync" : "d"})]


def f(x, filt):
    for content in x:
        if content['name']==filt:
            print ("test")
        yield content

def g(x,old, new):
    for content in x:
        if content["name"]==old:
            content["name"]=new
        yield content

def h(x, which):
    for content in x:
        if random.random()>0.5:
            content[which]=random.randint(0,100)
        yield content



p_f = partial(f, filt="Albert")
p_g = partial(g, old="Yves", new="Yeti")
p_h = partial(h, which='id')
iterator=(d for d in data_source)
for result in p_f(p_g(p_h(iterator))):
    print (result)

Which outputs:

OrderedDict([('id', '1'), ('name', 'Tom'), ('sync', 'a')])
OrderedDict([('id', 57), ('name', 'Steve'), ('sync', 'a')])
OrderedDict([('id', '3'), ('name', 'Ulrich'), ('sync', 'b')])
OrderedDict([('id', '4'), ('name', 'Victor'), ('sync', 'b')])
OrderedDict([('id', 33), ('name', 'Wolfgang'), ('sync', 'c')])
OrderedDict([('id', '6'), ('name', 'Xavier'), ('sync', 'c')])
OrderedDict([('id', 83), ('name', 'Yeti'), ('sync', 'c')])
OrderedDict([('id', '8'), ('name', 'Zaphod'), ('sync', 'd')])
test
OrderedDict([('id', '9'), ('name', 'Albert'), ('sync', 'd')])

But I want to be able to compose that function early - and bind the iterator of the composed function later.

Something like:

p_compiled = p_f(p_g(p_h))
for result in p_compiled(iterator):
    print (result)

But when I do this, I get TypeError: 'generator' object is not callable.

回答1:

It sounds like you just want a compose() function:

def compose(f, g):
    return lambda x: f(g(x))

p_compiled = compose(p_f, compose(p_g, p_h))
for result in p_compiled(iterator):
    print (result)

来源：https://stackoverflow.com/questions/56431391/transmit-parameter-to-the-inner-most-call-of-a-composed-function

标签

python

functional-programming