Difference between two rows in Spark dataframe

Question

I created a dataframe in Spark, by groupby column1 and date and calculated the amount.

val table = df1.groupBy($\"column1\",$\"date\").sum(\"amount\")
         


        

Answer 1

(table.filter($"Date".isin("1-jul", "1-dec"))
      .groupBy("Column1")
      .pivot("Date")
      .agg(first($"Amount"))
      .withColumn("diff", $"1-dec" - $"1-jul")
).show
+-------+-----+-----+----+
|Column1|1-dec|1-jul|diff|
+-------+-----+-----+----+
|      B|  300|  100| 200|
|      A| 3000| 1000|2000|
+-------+-----+-----+----+