Read a small random sample from a big CSV file into a Python data frame

Question

The CSV file that I want to read does not fit into main memory. How can I read a few (~10K) random lines of it and do some simple statistics on the selected data frame?

Answer 1

For example, you have the loan.csv, you can use this script to easily load the specified number of random items.

data = pd.read_csv('loan.csv').sample(10000, random_state=44)