问题
I’m trying to create a udf that will replace negative values in a column with 0.
My dataframe is – called df, and contains one column called avg_x. This is my code for creating a udf
val noNegative = udf {(avg_acc_x: Double) => if(avg_acc_x < 0) 0 else "avg_acc_x"}
I get this error
java.lang.UnsupportedOperationException: Schema for type Any is not supported
df.printSchema returns
|-- avg_acc_x: double (nullable = false)
so I don’t understand why this error is occurring?
回答1:
It's because of the else returning a String: "avg_acc_x". Take away the quotes:
val noNegative = udf {(avg_acc_x: Double) => if(avg_acc_x < 0) 0 else avg_acc_x}
回答2:
This error basically occurs when the Analyzer cannot resolve the type of udf properly. That is, the types are mixed (Int and String).
Therefore,
if(avg_acc_x < 0) 0 else avg_acc_x (Integer Type)
or
if(avg_acc_x < 0) "0" else "avg_acc_x" (String Type)
should resolve this exception.
来源：https://stackoverflow.com/questions/37392900/spark-udf-error-schema-for-type-any-is-not-supported