PySpark - rename more than one column using withColumnRenamed

Question

I want to change names of two columns using spark withColumnRenamed function. Of course, I can write:

data = sqlCont         


        

Answer 1

Easiest way to do this is as follows:

Explanation:

1. Get all columns in the pyspark dataframe using df.columns
2. Create a list looping through each column from step 1
3. The list will output:col("col1").alias("col1_x").Do this only for the required columns
4. *[list] will unpack the list for select statement in pypsark

from pyspark.sql import functions as F (df .select(*[F.col(c).alias(f"{c}_x") for c in df.columns]) .toPandas().head() )

Hope this helps