I\'ve come across the glom() method on RDD. As per the documentation
glom()
Return an RDD created by coalescing all elements within each partition
glom() transforms each partition into a tuple (immutabe list) of elements. It creates an RDD of tuples. One tuple per partition.
RDD