How do I configure Vert.x event bus to work across cluster of Docker containers?

Question

In my current setup, I\'m using the default multicast option of the Hazelcast cluster manager. When I link the instances of my containerized Vertx modules (via Docker networ

Answer 1

One answer is here https://groups.google.com/d/msg/vertx/_2MzDDowMBM/nFoI_k6GAgAJ

I see this question come up a lot, and what a lot of people miss in the documentation (myself included) is that Event Bus does not use the cluster manager to send event bus messages. I.e. in your example with Hazelcast as the cluster manager, you have the Hazelcast cluster up and communicating properly (so your Cluster Manager is fine); however, the Event bus is failing to communicate with your other docker instances due to one or more of the following:

1. It is attempting to use an incorrect IP address to the other node (i.e. the IP of the private interface on the Docker instance, not the publicly mapped one)
2. It is attempting to communicate on a port Docker is not configured to forward (the event bus picks a dynamic port if you don't specify one)

What you need to do is:

1. Tell Vertx the IP address that the other nodes should use to talk to each instance ( using the -cluster-host [command line] , setClusterPublicHost [VertXOptions] or "vertx.cluster.public.host" [System Property] options)
2. Tell Vertx explicitly the Port to use for event bus communication and ensure Docker is forwarding traffic for those ports ( using the "vertx.cluster.public.port" [System Property], setClusterPublicPort [VertXOptions] or -cluster-port [command line] options). In the past, I have used 15701 because it is easy to remember (just a '1' in fromt of the Hazelcast ports).

The Event bus only uses the Cluster Manager to manage the IP/Port information of the other Vertx Instances and the registration of the Consumers/Producers. The communications are done independently of the cluster manager, which is why you can have the cluster manager configured properly and communicating, but still have no Event bus communications.

You may not need to do both the steps above if both your containers are running on the same host, but you definitely will once you start running them on separate hosts.

Something what also can happen, is that vert.x uses the loopback interface, when not specifying the IP which vert.x (not hazelcast) should take to communicate over eventbus. The problem here is, that you don't know which interface is taken to communicate over (loopback, interface with IP, you could even have multiple interfaces with IP).

To overcome this problem, I wrote a method once https://github.com/swisspush/vertx-cluster-watchdog/blob/master/src/main/java/org/swisspush/vertx/cluster/ClusterWatchdogRunner.java#L101