PySpark: dynamic union of DataFrames with different columns
问题 Consider the arrays as shown here. I have 3 sets of array: Array 1: C1 C2 C3 1 2 3 9 5 6 Array 2: C2 C3 C4 11 12 13 10 15 16 Array 3: C1 C4 111 112 110 115 I need the output as following, the input I can get any one value for C1, ..., C4 but while joining I need to get correct values and if the value is not there then it should be zero. Expected output: C1 C2 C3 C4 1 2 3 0 9 5 6 0 0 11 12 13 0 10 15 16 111 0 0 112 110 0 0 115 I have written pyspark code but I have hardcoded the value for the