Spark SQL UDF with complex input parameter

Question

I\'m trying to use UDF with input type Array of struct. I have the following structure of data this is only relevant part of a bigger structure

|--investment         


        

Answer 1

I created a simple library which derives the necessary encoders for complex Product types based on the input type parameters.

https://github.com/lesbroot/typedudf

import typedudf.TypedUdf
import typedudf.ParamEncoder._

case class Foo(x: Int, y: String)
val fooUdf = TypedUdf((foo: Foo) => foo.x + foo.y.length)
df.withColumn("sum", fooUdf($"foo"))