most efficient I/O setup between Stata and Python (Pandas)

问题

I am using Stata to process some data, export the data in a csv file and load it in Python using the pandas read_csv function.

The problem is that everything is so slow. Exporting from Stata to a csv file takes ages (exporting in the dta Stata format is much faster), and loading the data via read_csv is also very slow. Using the read_stata pandas function is even worse.

I wonder is there are any other options? Like exporting a format other than csv? My csv dataset is approx 6-7 Gb large.

Any help appreciated

Thanks

回答1:

Pretty efficient pd.read_stata()/.to_stata(), see here

来源：https://stackoverflow.com/questions/29972946/most-efficient-i-o-setup-between-stata-and-python-pandas