Wait node.js until Logstash is ready using containers

Question

I have the following setup in docker-compose.yml:

....

logstash:
  container_name: logstash
  image: docker.elastic.co/logstash/logstash:6.2.4

node:
  container_name: node
  build:
    context: ./node/
    dockerfile: ./Dockerfile
  depends_on:
    - logstash

....

I'm using the package winston-logstash to wire them together.

This is the transport layer:

const logstashHost = process.env.LOGSTASH_HOST || 'logstash'
const logstashPort = process.env.LOGSTASH_PORT || 5045

new (winstonLogstash.Logstash)({
  host: logstashHost,
  port: logstashPort,
  node_name: 'node',
  timestamp: true,
  max_connect_retries: 5,
  timeout_connect_retries: 1000,
})

And the pipeline configuration:

input {
  tcp {
    port => 5045
  }
}
output {
  stdout{}
  elasticsearch {
    hosts => ["elasticsearch:9200"]
  }
}

Using docker-compose up results in Error: Max retries reached, transport in silent mode, OFFLINE

If I manually start the server either using a large setTimeout or incrementing the number of connection retries it finally works. It works too if I start logstash and after a while I start node container.

The problem is that obviously this is not a good practice, I can't guess how long logstash will take to start, and the depends_on directive inside docker-compose.yml doesn't help at all.

I need a way to know when logstash is ready and start the node container after that.

Answer 1

Docker compose does not wait until a container is ready, it will only wait until it's running.

depends_on will only ensure that logstash launches before your node container, but again, this doesn't mean it will wait until it is ready.

You can either handle the checks yourself on node, or use a wrapper script. In docker-compose documentation, they recommend, wait-for-it or dockerize

You can read more on this in here

---

Custom wrapper

Your node container command can change from node index.js (or whatever you have), to bash wait-for-logtash.sh:

#!/bin/bash

## Or whatever command is used for checking logstash availability
until curl 'http://logstash:5045' 2> /dev/null; do
  echo "Waiting for logtash..."
  sleep 1; 
done

# Start your server
node index.js