node and Error: EMFILE, too many open files

Question

For some days I have searched for a working solution to an error

Error: EMFILE, too many open files

It seems that many people have the same proble

Answer 1

Here's my two cents: Considering a CSV file is just lines of text I've streamed the data (strings) to avoid this problem.

Easiest solution for me that worked in my usecase.

It can be used with graceful fs or standard fs. Just note that there won't be headers in the file when creating.

// import graceful-fs or normal fs
const fs = require("graceful-fs"); // or use: const fs = require("fs") 

// Create output file and set it up to receive streamed data
// Flag is to say "append" so that data can be recursively added to the same file 
let fakeCSV = fs.createWriteStream("./output/document.csv", {
  flags: "a",
});

and the data that needs to be streamed to the file i've done like this

// create custom streamer that can be invoked when needed
const customStreamer = (dataToWrite) => {
  fakeCSV.write(dataToWrite + "\n");
};

Note that the dataToWrite is simply a string with a custom seperator like ";" or ",". i.e.

const dataToWrite = "batman" + ";" + "superman"
customStreamer(dataToWrite);

This writes "batman;superman" to the file.

---

• Note that there's no error catching or whatsoever in this example.
• Docs: https://nodejs.org/api/fs.html#fs_fs_createwritestream_path_options