DDP between two servers doesn't reconnect

Question

I have two meteor applications connected via DDP on different servers and server A send data to server B. This is the way they work.

Server A

Answer 1

Emily Stark, from the Meteor Team, confirmed that this is due to a missing feature on the current implementation (version 0.7.0.1 at the moment I write this answer). Their answer is here https://github.com/meteor/meteor/issues/1543. Below is their answer and a workaround she suggest:

The server-to-server connection is not reconnecting because Meteor currently doesn't do any heartbeating on server-to-server DDP connections. Just as in any other TCP connection, once you switch to a different router, no data can be sent or received on the connection, but the client will not notice unless it attempts to send some data and times out. This differs from browser-to-server DDP connections, which run over SockJS. SockJS does its own heartbeating that we can use to detect dead connections.

To see this in action, here is some code that I added to server-b in your example:

var heartbeatOutstanding = false;
Meteor.setInterval(function () {
  if (! heartbeatOutstanding) {
   console.log("Sending heartbeat");
    remote.call("heartbeat", function () {
      console.log("Heartbeat returned");
      heartbeatOutstanding = false;
    });
    heartbeatOutstanding = true;
  }
}, 3000);

remote.onReconnect = function () {
  console.log("RECONNECTING REMOTE");
};

With this code added in there, server-b will reconnect after a long enough time goes by without an ACK from server-a for the TCP segments that are delivering the heartbeat method call. On my machine, this is just a couple minutes, and I get an ETIMEDOUT followed by a reconnect.

I've opened a separate task for us to think about implementing heartbeating on server-to-server DDP connections during our next bug week. In the meantime, you can always implement heartbeating in your application to ensure that a DDP reconnection happens if the client can no longer talk to the server.