PySpark - Explode columns into rows based on the type of the column

Question

Given a Dataframe:

+---+-----------+---------+-------+------------+
| id|      score|tx_amount|isValid|    greeting|
+---+-----------+---------+-------+---------         


        

Answer 1

you can try several unions :

df = df.select(
    "id",
    F.col("score").cast("string").alias("col_value"),
    F.lit("Y").alias("is_score"),
    F.lit("N").alias("is_amount"),
    F.lit("N").alias("is_boolean"),
    F.lit("N").alias("is_text"),
).union(df.select(
    "id",
    F.col("tx_amount").cast("string").alias("col_value"),
    F.lit("N").alias("is_score"),
    F.lit("Y").alias("is_amount"),
    F.lit("N").alias("is_boolean"),
    F.lit("N").alias("is_text"),
)).union(...) # etc