Removing duplicate columns after a DF join in Spark

Question

When you join two DFs with similar column names:

df = df1.join(df2, df1[\'id\'] == df2[\'id\'])

Join works fine but you can\'t call the

Answer 1

In pyspark, you can join on multiple columns as per below

df = df1.join(df2, ['each', 'shared', 'col'], how='full')

Original answer from: How to perform union on two DataFrames with different amounts of columns in spark?