Pandas: Sampling a DataFrame [duplicate]

Answer 1

What version of pandas are you using? For me your code works fine (i`m on git master).

Another approach could be:

In [117]: import pandas

In [118]: import random

In [119]: df = pandas.DataFrame(np.random.randn(100, 4), columns=list('ABCD'))

In [120]: rows = random.sample(df.index, 10)

In [121]: df_10 = df.ix[rows]

In [122]: df_90 = df.drop(rows)

Newer version (from 0.16.1 on) supports this directly: http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.sample.html

Answer 2

Pandas 0.16.1 have a sample method for that.

Answer 3

I have found that np.random.choice() new in NumPy 1.7.0 works quite well for this.

For example you can pass the index values from a DataFrame and and the integer 10 to select 10 random uniformly sampled rows.

rows = np.random.choice(df.index.values, 10)
sampled_df = df.ix[rows]

Answer 4

New in version 0.16.1:

sample_dataframe = your_dataframe.sample(n=how_many_rows_you_want)

doc here: http://pandas.pydata.org/pandas-docs/version/0.17.0/generated/pandas.DataFrame.sample.html

Answer 5

If you're using pandas.read_csv you can directly sample when loading the data, by using the skiprows parameter. Here is a short article I've written on this - https://nikolaygrozev.wordpress.com/2015/06/16/fast-and-simple-sampling-in-pandas-when-loading-data-from-files/