I am trying to filter a DataFrame by keeping only those rows that have a certain string column non-empty.
The operation is the following:
df.filter($"stringColumn" !== "")
My compiler shows that the !== is deprecated since I moved to Spark 2.0.1
How can I check if a string column value is empty in Spark > 2.0?
Use =!=
as a replacement:
df.filter($"stringColumn" =!= "")
来源:https://stackoverflow.com/questions/40154104/spark-2-0-dataframe-filter-a-string-column-unequal-operator-is-deprecat