My docker-compose.yml is
version: "2.1" services: pg-master: image: postgres-9.6:3 ports: - "5432:5432" environment: - VIRTUAL_HOST=postgres.local web: build: . links: - pg-master:postgres.local ports: - "8080:8080" zookeeper: image: jplock/zookeeper:3.4.6 ports: - "2181:2181" kafka: image: wurstmeister/kafka:1.0.0 ports: - "9092:9092" environment: - KAFKA_ADVERTISED_HOST_NAME=localhost - KAFKA_ADVERTISED_PORT=9092 - KAFKA_CREATE_TOPICS=TestTopic1:3:1 - KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181 - KAFKA_BROKER_ID=999 volumes: - /var/run/docker.sock:/var/run/docker.sock In web docker container, the java kafka producer:
Properties props = new Properties(); props.put("bootstrap.servers", "localhost:9092"); props.put("acks", "all"); props.put("retries", 0); props.put("batch.size", 16384); props.put("linger.ms", 1); props.put("buffer.memory", 33554432); props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer"); props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer"); this.kafkaProducer = new KafkaProducer<String, String>(props); But, I got [kafka-producer-network-thread | producer-2] WARN org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-2] Connection to node -1 could not be established. Broker may not be available.
NO idea why. any suggestions welcomed. Thanks very much.
UPDATE
I can produce and consume messages by bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic TestTopic1 --from-beginning and bin/kafka-console-producer.sh --broker-list localhost:9092 --topic TestTopic1 inside and outside kafka docker container.
what is localhost inside docker container?
UPDATE
Now, I changed the KAFKA_ADVERTISED_HOST_NAME and bootstrap.servers of Kafka Java producer config manually to the docker host ip (got by ifconfig). Now, it is working. Now, I have no idea how to get docker host ip and set it in docker-compose files automatically by program or bash.