Leader Not Available Kafka in Console Producer

Question

I am trying to use Kafka.
All configurations are done properly but when I try to produce message from console I keep getting the following error

WARN Err         


        

Answer 1

When LEADER_NOT_AVAILABLE error throws, just restart the kafka broker:

/bin/kafka-server-stop.sh

followed by

/bin/kafka-server-start.sh config/server.properties

(Note: Zookeeper must be running by this time ,if you do otherway it wont work )

Answer 2

This below line I have added in config/server.properties, that resolved my issue similar above issue. Hope this helps, its pretty much well documented in server.properties file, try to read and understand before you modify this. advertised.listeners=PLAINTEXT://<your_kafka_server_ip>:9092

Answer 3

For me, it was happen due to a miss configuration
Docker port (9093)
Kafka command port "bin/kafka-console-producer.sh --broker-list localhost:9092 --topic TopicName"
I checked my configuration to match port and now everything is ok

Answer 4

Adding this since it may help others. A Common problem can be a misconfiguration of advertised.host.name. With Docker using docker-compose setting the name of the service inside KAFKA_ADVERTISED_HOST_NAME wont work unless you set the hostname as well. docker-compose.yml example:

  kafka:
    image: wurstmeister/kafka
    ports:
      - "9092:9092"
    hostname: kafka
    environment:
      KAFKA_ADVERTISED_HOST_NAME: kafka
      KAFKA_CREATE_TOPICS: "test:1:1"
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

The above without hostname: kafka can issue a LEADER_NOT_AVAILABLE when trying to connect. You can find an example of a working docker-compose configuration here

Answer 5

If you get repeated error messages like this:

Error while fetching metadata with correlation id 3991 : {your.topic=LEADER_NOT_AVAILABLE}

Or

Discovered group coordinator 172.25.1.2:9092 (id: 2147483645 rack: null) (org.apache.kafka.clients.consumer.internals.AbstractCoordinator:677)
(Re-)joining group (org.apache.kafka.clients.consumer.internals.AbstractCoordinator:509)
Group coordinator 172.25.1.2:9092 (id: 2147483645 rack: null) is unavailable or invalid, will attempt rediscovery (org.apache.kafka.clients.consumer.internals.AbstractCoordinator:729)
Discovered group coordinator 172.25.40.219:9092 (id: 2147483645 rack: null) (org.apache.kafka.clients.consumer.internals.AbstractCoordinator:677)
Group coordinator 172.25.1.2:9092 (id: 2147483645 rack: null) is unavailable or invalid, will attempt rediscovery (org.apache.kafka.clients.consumer.internals.AbstractCoordinator:729)

Then, you need to configure listener settings like this in the kafka server.properties:

 listeners=PLAINTEXT://your.server.ip:9092

This is the solution tried on Apacke Kafka 2.5.0 and confluent platform 5.4.1.

Answer 6

For me, the cause was using a specific Zookeeper that was not part of the Kafka package. That Zookeeper was already installed on the machine for other purposes. Apparently Kafka does not work with just any Zookeeper. Switching to the Zookeeper that came with Kafka solved it for me. To not conflict with the existing Zookeeper, I had to modify my confguration to have the Zookeeper listen on a different port:

[root@host /opt/kafka/config]# grep 2182 *
server.properties:zookeeper.connect=localhost:2182
zookeeper.properties:clientPort=2182