How to convert empty arrays to nulls?

Question

I have below dataframe and i need to convert empty arrays to null.

+----+---------+-----------+
|  id|count(AS)|count(asdr)|
+----+---------+-----------+
|11         


        

Answer 1

You need to check for the size of the array type column. Like:

df.show()
+----+---+
|  id|arr|
+----+---+
|1110| []|
+----+---+

df.withColumn("arr", when(size(col("arr")) == 0 , lit(None)).otherwise(col("arr") ) ).show()

+----+----+
|  id| arr|
+----+----+
|1110|null|
+----+----+