Spark Dataframe - Best way to cogroup dataframes

◇◆丶佛笑我妖孽 提交于 2019-12-10 16:53:50

问题


I currently load CSV files into Dataframes using the databricks library.

I'm looking for the best generic approach to cogroup my loaded dataframes using a specific key since the cogroup operation is only available for PairRDDs.

I found this post which implements a cogroup feature for Dataframes but I guess there are some different approaches :

https://gist.github.com/ahoy-jon/b65754cde98cc48b9b38

Have you please ever faced this situation ?

Thanks.

来源:https://stackoverflow.com/questions/31806473/spark-dataframe-best-way-to-cogroup-dataframes

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!