I often have the need to perform custom aggregations on dataframes in spark 2.1, and used these two approaches :