Zookeeper connection error

Question

We have a standalone zookeeper setup on a dev machine. It works fine for every other dev machine except this one testdev machine.

We get this error over and over aga

Answer 1

I also ran into this problem last week and have managed to fix this now. I got the idea to resolve this one from the response shared by @gukoff.

My requirement and situation was slightly different from the ones shared so far but the issue was fundamentally the same so I thought of sharing it on this thread.

I was actually trying to query zookeeper quorum (after every 30 seconds) for some information from my application and was using the Curator Framework for this purpose (the methods available in LeaderLatch class). So, essentially I was starting up a CuratorFramework client and supplying this to LeaderLatch object.

Only after I ran into the error mentioned in this thread - I realised that I did not close the zookeeper client connection(s) established in my applications. The maxClientCnxns property had the value of 60 and as soon as the number of connections (all of them were stale connections) touched 60, my application started complaining with this error.

I found out about the number of open connections by:

1. Checking the zookeeper logs, where there were warning messages stating "Too many connections from {IP address of the host}"

2. Running the following netstat command from the same host mentioned in the above logs where my application was running:

netstat -no | grep :2181 | wc -l

Note: The 2181 port is the default for zookeeper supplied as a parameter in grep to match the zookeeper connections.

To fix this, I cleared up all of those stale connections manually and then added the code for closing the zookeeper client connections gracefully in my application.

I hope this helps!