how to populate select clause of dataframe dynamically? giving AnalysisException

Question

I am Using spark-sql 2.4.1 and java 8.

 val country_df = Seq(
    (\"us\",2001),
    (\"fr\",2002),
    (\"jp\",2002),
    (\"in\",2001),
    (\"fr\",2003),
         


        

Answer 1

scala> val colname = col_df.rdd.collect.toList.map(x => x(0).toString).toSeq

scala> data_df.select(colname.head, colname.tail: _*).show()
+----------+----------+
|        us|        in|
+----------+----------+
|us_state_1|in_state_1|
|us_state_2|in_state_2|
|us_state_3|in_state_3|
+----------+----------+