Spark 2.0, DataFrame, filter a string column, unequal operator (!==) is deprecated

问题

I am trying to filter a DataFrame by keeping only those rows that have a certain string column non-empty.

The operation is the following:

df.filter($"stringColumn" !== "")

My compiler shows that the !== is deprecated since I moved to Spark 2.0.1

How can I check if a string column value is empty in Spark > 2.0?

回答1:

Use =!= as a replacement:

df.filter($"stringColumn" =!= "")

来源：https://stackoverflow.com/questions/40154104/spark-2-0-dataframe-filter-a-string-column-unequal-operator-is-deprecat