Reading CSV files in a loop using pandas, then concatenating them

Question

I have 10 csv files, named data_run1_all.csv, data_run2_all.csv, ..., data_run10_all.csv. CSV files have same columns, but different r

Answer 1

Read your CSVs in a loop and call pd.concat:

file_name = 'data_run{}_all.csv'
df_list = []
for i in range(1, 11):
    df_list.append(pd.read_csv(file_name.format(i))

df = pd.concat(df_list)

---

Alternatively, you could build the list inside a comprehension:

file_name = 'data_run{}_all.csv'
df = pd.concat([pd.read_csv(file_name.format(i)) for i in range(1, 11)])

Answer 2

You need to make df_run a list. You could do something like this:

df_run = []
for i in range(1,10):
  df_run.append(pandas.read_csv('data_run{0}_all.csv'.format(i))
for df in df_run:
  // Do your processing

Or do everything in a single loop, and avoid having the list.