How to slice a pyspark dataframe in two row-wise

后端 未结 5 578
走了就别回头了
走了就别回头了 2020-12-09 18:58

I am working in Databricks.

I have a dataframe which contains 500 rows, I would like to create two dataframes on containing 100 rows and the other containing the rem

5条回答
  •  被撕碎了的回忆
    2020-12-09 19:27

    Providing a much less complicated solution here more similar to what was requested:

    (Works in Spark 2.4 +)

    # Starting
    print('Starting row count:',df.count())
    print('Starting column count:',len(df.columns))
    
    # Slice rows
    df2 = df.limit(3)
    print('Sliced row count:',df2.count())
    
    # Slice columns
    cols_list = df.columns[0:1]
    df3 = df.select(cols_list)
    print('Sliced column count:',len(df3.columns))
    

提交回复
热议问题