check if a row value is null in spark dataframe

Question

I am using a custom function in pyspark to check a condition for each row in a spark dataframe and add columns if condition is true.

The code is as below:

Answer 1

Considering that sdf is a DataFrame you can use a select statement.

sdf.select("*", when(col("pro").isNull(), lit("new pro")).otherwise(col("pro")))