How to gunzip stream in nodejs?

问题

I'm trying to accomplish quite an easy task but I'm a little bit confused and got stuck on using zlib in nodejs. I'm building functionality that includes me downloading file from aws S3 which is gziped, unzipping it and reading it line by line. I want to accomplish all of this using streams as I believe it is possible to do so in nodejs.

Here is my current code base:

//downloading zipped file from aws s3:
//params are configured correctly to access my aws s3 bucket and file

s3.getObject(params, function(err, data) {
  if (err) {
    console.log(err);
  } else {

    //trying to unzip received stream:
    //data.Body is a buffer from s3
    zlib.gunzip(data.Body, function(err, unzippedStream) {
      if (err) {
        console.log(err);
      } else {

        //reading line by line unzziped stream:
        var lineReader = readline.createInterface({
          input: unzippedStream
        });
        lineReader.on('line', function(lines) {
          console.log(lines);
        });
      }
    });
  }
});

I get an error saying:

 readline.js:113

        input.on('data', ondata);
              ^

    TypeError: input.on is not a function

I believe a problem might be in unzipping process, but I'm not too sure what is wrong, any help would be appreciated.

回答1:

I don't have an S3 account to test with, but reading the docs suggests that s3.getObject() can return a stream, in which case I think that this might work:

var lineReader = readline.createInterface({
  input: s3.getObject(params).pipe(zlib.createGunzip())
});
lineReader.on('line', function(lines) {
  console.log(lines);
});

EDIT: looks like the API may have changed, and you're now required to instantiate a stream object manually before you can pipe it through anything else:

s3.getObject(params).createReadStream().pipe(...)

来源：https://stackoverflow.com/questions/38120231/how-to-gunzip-stream-in-nodejs

标签

node.js

stream

zlib

gunzip