Folder containing 400 mb of files takes 4,2 gb

Question

I\'m trying to handle large amounts of data and get them ready for my LSTM. Since my data contains 1M rows and for each prediction I have to get the past 24h of data, I\'m s