Leader Not Available Kafka in Console Producer

Question

I am trying to use Kafka.
All configurations are done properly but when I try to produce message from console I keep getting the following error

WARN Err         


        

Answer 1

I tried all the recommendations listed here. What worked for me was to go to server.properties and add:

port = 9092
advertised.host.name = localhost 

Leave listeners and advertised_listeners commented out.

Answer 2

In my case, it was working fine at home, but it was failing in office, the moment I connect to office network.

So modified the config/server.properties listeners=PLAINTEXT://:9092 to listeners=PLAINTEXT://localhost:9092

In my case, I was getting while describing the Consumer Group

Answer 3

I had kafka running as a Docker container and similar messages were flooding to the log.
And KAFKA_ADVERTISED_HOST_NAME was set to 'kafka'.

In my case the reason for error was the missing /etc/hosts record for 'kafka' in 'kafka' container itself.
So, for example, running ping kafka inside 'kafka' container would fail with ping: bad address 'kafka'

In terms of Docker this problem gets solved by specifying hostname for the container.

Options to achieve it:

• docker run --hostname ...
• docker run -it --add-host ...
• hostname in docker-compose
• hostname in AWS EC2 Task Definition

Answer 4

We tend to get this message when we try to subscribe to a topic that has not been created yet. We generally rely on topics to be created a priori in our deployed environments, but we have component tests that run against a dockerized kafka instance, which starts clean every time.

In that case, we use AdminUtils in our test setup to check if the topic exists and create it if not. See this other stack overflow for more about setting up AdminUtils.

Answer 5

Since I wanted my kafka broker to connect with remote producers and consumers, So I don't want advertised.listener to be commented out. In my case, (running kafka on kubernetes), I found out that my kafka pod was not assigned any Cluster IP. By removing the line clusterIP: None from services.yml, the kubernetes assigns an internal-ip to kafka pod. This resolved my issue of LEADER_NOT_AVAILABLE and also remote connection of kafka producers/consumers.

Answer 6

The advertised listeners as mentioned in the above answers could be one of the reason. The other possible reasons are:

1. The topic might not have been created. You can check this using bin/kafka-topics --list --zookeeper <zookeeper_ip>:<zookeeper_port>
2. Check your bootstrap servers that you have given to the producer to fetch the metadata. If the bootstrap server does not contain the latest metadata about the topic (for example, when it lost its zookeeper claim). You must be adding more than one bootstrap servers.

Also, ensure that you have the advertised listener set to IP:9092 instead of localhost:9092. The latter means that the broker is accessible only through the localhost.

When I encountered the error, I remember to have used PLAINTEXT://<ip>:<PORT> in the list of bootstrap servers (or broker list) and it worked, strangely.

bin/kafka-console-producer --topic sample --broker-list PLAINTEXT://<IP>:<PORT>