How do I read the contents of a Node.js stream into a string variable?

Question

I\'m hacking on a Node program that uses smtp-protocol to capture SMTP emails and act on the mail data. The library provides the mail data as a stream, and I don\'t know how

Answer 1

setEncoding('utf8');

Well done Sebastian J above.

I had the "buffer problem" with a few lines of test code I had, and added the encoding information and it solved it, see below.

Demonstrate the problem

software

// process.stdin.setEncoding('utf8');
process.stdin.on('data', (data) => {
    console.log(typeof(data), data);
});

input

hello world

output

object <Buffer 68 65 6c 6c 6f 20 77 6f 72 6c 64 0d 0a>

Demonstrate the solution

software

process.stdin.setEncoding('utf8'); // <- Activate!
process.stdin.on('data', (data) => {
    console.log(typeof(data), data);
});

input

hello world

output

string hello world

Answer 2

I had more luck using like that :

let string = '';
readstream
    .on('data', (buf) => string += buf.toString())
    .on('end', () => console.log(string));

I use node v9.11.1 and the readstream is the response from a http.get callback.

Answer 3

The cleanest solution may be to use the "string-stream" package, which converts a stream to a string with a promise.

const streamString = require('stream-string')

streamString(myStream).then(string_variable => {
    // myStream was converted to a string, and that string is stored in string_variable
    console.log(string_variable)

}).catch(err => {
     // myStream emitted an error event (err), so the promise from stream-string was rejected
    throw err
})

Answer 4

This worked for me and is based on Node v6.7.0 docs:

let output = '';
stream.on('readable', function() {
    let read = stream.read();
    if (read !== null) {
        // New stream data is available
        output += read.toString();
    } else {
        // Stream is now finished when read is null.
        // You can callback here e.g.:
        callback(null, output);
    }
});

stream.on('error', function(err) {
  callback(err, null);
})

Answer 5

None of the above worked for me. I needed to use the Buffer object:

  const chunks = [];

  readStream.on("data", function (chunk) {
    chunks.push(chunk);
  });

  // Send the buffer or you can put it into a var
  readStream.on("end", function () {
    res.send(Buffer.concat(chunks));
  });

Answer 6

From the nodejs documentation you should do this - always remember a string without knowing the encoding is just a bunch of bytes:

var readable = getReadableStreamSomehow();
readable.setEncoding('utf8');
readable.on('data', function(chunk) {
  assert.equal(typeof chunk, 'string');
  console.log('got %d characters of string data', chunk.length);
})