Python Spark Cumulative Sum by Group Using DataFrame

Question

How do I compute the cumulative sum per group specifically using the DataFrame abstraction; and in PySpark?

With an example da

Answer 1

I have tried this way and it worked for me.

from pyspark.sql import Window

from pyspark.sql import functions as f

import sys

cum_sum = DF.withColumn('cumsum', f.sum('value').over(Window.partitionBy('class').orderBy('time').rowsBetween(-sys.maxsize, 0)))
cum_sum.show()