Spark: Convert column of string to an array

Question

How to convert a column that has been read as a string into a column of arrays? i.e. convert from below schema

scala> test.printSchema
root
 |-- a: long (         


        

Answer 1

There are various method,

The best way to do is using split function and cast to array

data.withColumn("b", split(col("b"), ",").cast("array"))

You can also create simple udf to convert the values

val tolong = udf((value : String) => value.split(",").map(_.toLong))

data.withColumn("newB", tolong(data("b"))).show

Hope this helps!