what is difference between file reading and streaming?

Question

I read in some book that using streaming is better than reading a whole file at a time in node.js, I understand the idea .. but I wonder isn\'t file reading

Answer 1

So what's the difference here ?? also what is the difference between fs.createReadStream(); and fs.readFile(); both are asynchronous, right !!

Well aside from the fact that fs.createReadStream() directly returns a stream object, and fs.readFile() expects a callback function in the second argument, there is another huge difference.

Yes they are both asynchronous, but that doesn't change the fact that fs.readFile() doesn't give you any data until the entire file has been buffered into memory. This is much less memory-efficient and slower when relaying data back through server responses. With fs.createReadStream(), you can pipe() the stream object directly to a server's response object, which means your client can immediately start receiving data even if the file is 500MB.

Not only this, you also improve memory efficiency by dealing with the file one chunk at a time rather than all at once. This means that your memory only has to buffer the file contents a few kilobytes at a time rather than all at once.

Here's two snippets demonstrating what I'm saying:

const fs = require('fs');
const http = require('http');

// using readFile()
http.createServer(function (req, res) {
    // let's pretend this is a huge 500MB zip file
    fs.readFile('some/file/path.zip', function (err, data) {
        // entire file must be buffered in memory to data, which could be very slow
        // entire chunk is sent at once, no streaming here
        res.write(data);
        res.end();
    });
});

// using createReadStream()
http.createServer(function (req, res) {
    // processes the large file in chunks
    // sending them to client as soon as they're ready
    fs.createReadStream('some/file/path.zip').pipe(res);
    // this is more memory-efficient and responsive
});