Elasticsearch in Docker container cluster

Question

I want to run 2 instances of Elasticsearch on 2 different hosts.

I have built my own Docker image based on Ubuntu 14.04 and the 1.3.2 version of Elasticsearch. If I r

Answer 1

You should be able to communicate the two containers running in different hosts as far as the host machines are accessible between them in the ports needed. I think your problem is that you are trying to use ElasticSearch multicast discovery, but if then you need to expose also port 54328 of the containers. If it doesn't work you can also try to configure ElasticSearch using unicast, setting adequately the machines IP's in your elasticsearch.yml.

Answer 2

Using docker-compose is much easier than running it manually in command line:

elasticsearch_master:
    image: elasticsearch:latest
    command: "elasticsearch -Des.cluster.name=workagram -Des.node.master=true -Des.node.data=false"
    environment:
       - ES_HEAP_SIZE=512m
    ports:
      - "9200:9200"
      - "9300:9300"

elasticsearch1:
    image: elasticsearch:latest
    command: "elasticsearch -Des.cluster.name=workagram -Des.discovery.zen.ping.unicast.hosts=elasticsearch_master"
    links:
      - elasticsearch_master
    volumes:
      - "/opt/elasticsearch/data"
    environment:
       - ES_HEAP_SIZE=512m
elasticsearch2:
    image: elasticsearch:latest
    command: "elasticsearch -Des.cluster.name=workagram -Des.discovery.zen.ping.unicast.hosts=elasticsearch_master"
    links:
      - elasticsearch_master
    volumes:
      - "/opt/elasticsearch/data"
    environment:
       - ES_HEAP_SIZE=512m

Answer 3

I was able to get clustering working using unicast across two docker hosts. I just happen to be using the ehazlett/elasticsearch image, but I do not think this should matter all that much. The really important bit seems to be setting the network.publish_host setting to a public or routable IP its docker host.

Configuration

---

docker-host-01

eth0: 192.168.1.10
Docker version 1.4.1, build 5bc2ff8/1.4.1

docker-host-02

eth0: 192.168.1.20
Docker version 1.4.1, build 5bc2ff8/1.4.1

Building the Cluster

---

On Docker Host 01

docker run -d \
  -p 9200:9200 \
  -p 9300:9300 \
  ehazlett/elasticsearch \
  --cluster.name=unicast \
  --network.publish_host=192.168.1.10 \
  --discovery.zen.ping.multicast.enabled=false \
  --discovery.zen.ping.unicast.hosts=192.168.1.20 \
  --discovery.zen.ping.timeout=3s \
  --discovery.zen.minimum_master_nodes=1

On Docker Host 02

docker run -d \
  -p 9200:9200 \
  -p 9300:9300 \
  ehazlett/elasticsearch \
  --cluster.name=unicast \
  --network.publish_host=192.168.1.20 \
  --discovery.zen.ping.multicast.enabled=false \
  --discovery.zen.ping.unicast.hosts=192.168.1.10 \
  --discovery.zen.ping.timeout=3s \
  --discovery.zen.minimum_master_nodes=1