Adding a group count column to a PySpark dataframe

Question

I am coming from R and the tidyverse to PySpark due to its superior Spark handling, and I am struggling to map certain concepts from one context to the other.

In par

Answer 1

Great answer @David Bruce Borenstein,

I found we can get even more close to the tidyverse example:

from pyspark.sql import Window
w = Window.partitionBy('x')
df.withColumn('n', f.count('x').over(w)).sort('x', 'y').show()