Filter by whether column value equals a list in Spark

Question

I\'m trying to filter a Spark dataframe based on whether the values in a column equal a list. I would like to do something like this:

filtered_df = df.where(         


        

Answer 1

You might create a udf. For example:

def test_in(x):
    return x == ['list','of' , 'stuff']

from pyspark.sql.functions import udf
f = udf(test_in, pyspark.sql.types.BooleanType())
filtered_df = df.where(f(df.a))