I am working in Databricks.
I have a dataframe which contains 500 rows, I would like to create two dataframes on containing 100 rows and the other containing the rem
Try by this way :
df1_list = df.collect()[:99] #this will return list df1 = spark.createDataFrame(df1) #convert it to spark dataframe
similarly for this as well:
df2_list = df.collect()[100:499] df2 = spark.createDataFrame(df2)