Docker Kafka w/ Python consumer

Question

I am using dockerized Kafka and written one Kafka consumer program. It works perfectly when I run Kafka in docker and application at my local machine. But when I configured

Answer 1

This line

KAFKA_ADVERTISED_HOST_NAME: localhost

Says the broker is advertising itself as being available only on localhost, which means all Kafka clients would only get back itself, not the actual list of real broker addresses. This would be fine if your clients are only located on your host - requests always go to localhost, which is forwarded to the container.

But, for apps in other containers, they need to point at the Kafka container, so it should say KAFKA_ADVERTISED_HOST_NAME: kafka, where kafka here is the name of the Docker Compose Service. Then clients in other containers would try to connect to that one

---

That being said, then, this line

consumer = KafkaConsumer('test', bootstrap_servers='localhost:9092')

You are pointing the Python container at itself, not the kafka container.

It should say kafka:9092 instead