kafka broker not available at starting

Question

I set on a ubuntu node of a cluster a kafka 0.11.0.0 instance. Until some weeks ago everything worked fine, today I\'m trying to starting it and I obtain this error after th

Answer 1

With spring-boot, kafka (image: spotify/kafka) and docker-compose combination, you might need to set below property in project:

broker-addresses: <hostname>:9092

Also, the services in docker-compose need to have environment property configured as below,

KAFKA_BOOTSTRAP_SERVERS: <hostname>:9092

Hope this might helps someone.

Answer 2

We faced the same situation, when we started to search the logs for the actual cause of the error.

When we were starting the Kafka cluster, it was using by default

listeners=PLAINTEXT://:9092

to connect and was not able to find the host.

We changed the line to

listeners=PLAINTEXT://<our ip address> :9092,

then

listeners=PLAINTEXT://<our ip address> :9093, 

in our server-1.properties ( another broker id file ) and the same we replicated in other server.properties file and restarted our cluster.

Answer 3

i configed the kafka server.properties's listener as listeners=PLAINTEXT://**10.127.96.151**:9092

but i request the consumer as ./kafka-console-consumer.sh --bootstrap-server **localhost**:9092 --topic topic1 --from-beginning , then it bursts the WARNs.

when i fix the consumer bootstrap-server as 10.127.96.151:9092, it runs all well with no more WARNs

Answer 4

I server.propertes:

listeners=PLAINTEXT://hidden_ip:9092

when I run:
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic lt1 bin/kafka-console-producer.sh --broker-list localhost:9092 --topic lt1
I get same error just like you get.and I try run :

bin/kafka-console-producer.sh --broker-list hidden_ip:9092 --topic lt1

it work,I get no error.
So i think you should check what's port kafka server used.This error when port is not esestablished or you connect wrong ip.
PS:I run this on same machine.

Answer 5

error message: "Connection to node -1 could not be established. Broker may not be available"

Solution:

I was getting this error because I was having a different version of kafka running on cluster vs local. Make sure the version is the same. I was specifying the version in pom.xml under properties tag

Answer 6

If you are using Spring Boot, please see if the below property is set in application.properties file, assuming a producer application only:

spring.kafka.producer.bootstrap-servers=localhost:9092

Otherwise, the following should work in general:

kafka.bootstrap.servers=localhost:9200

I didn’t have to change listeners in the server.properties file, as others have proposed.