I am working in Databricks.
I have a dataframe which contains 500 rows, I would like to create two dataframes on containing 100 rows and the other containing the rem
Providing a much less complicated solution here more similar to what was requested:
(Works in Spark 2.4 +)
# Starting
print('Starting row count:',df.count())
print('Starting column count:',len(df.columns))
# Slice rows
df2 = df.limit(3)
print('Sliced row count:',df2.count())
# Slice columns
cols_list = df.columns[0:1]
df3 = df.select(cols_list)
print('Sliced column count:',len(df3.columns))