Spark Streaming Kafka Integration direct Approach EOFException

问题

when i run spark streaming example org.apache.spark.examples.streaming.JavaDirectKafkaWordCount,i caught an EOFException follow,how can I resolve it

Exception in thread "main" org.apache.spark.SparkException: java.io.EOFException: Received -1 when reading from channel, socket has likely been closed.
java.io.EOFException: Received -1 when reading from channel, socket has likely been closed.
java.io.EOFException: Received -1 when reading from channel, socket has likely been closed.
    at org.apache.spark.streaming.kafka.KafkaUtils$$anonfun$createDirectStream$2.apply(KafkaUtils.scala:413)
    at org.apache.spark.streaming.kafka.KafkaUtils$$anonfun$createDirectStream$2.apply(KafkaUtils.scala:413)
    at scala.util.Either.fold(Either.scala:97)
    at org.apache.spark.streaming.kafka.KafkaUtils$.createDirectStream(KafkaUtils.scala:412)
    at org.apache.spark.streaming.kafka.KafkaUtils$.createDirectStream(KafkaUtils.scala:528)
    at org.apache.spark.streaming.kafka.KafkaUtils.createDirectStream(KafkaUtils.scala)

回答1:

The direct stream uses a low level Kafka consumer underneath therefore one needs to provide the list of brokers. Most likely you haven't so you may want to set the metadata.broker.list property in the form bhost1:9092,bhost2:9092,...,bhostN:9092.

See also Kafka SimpleConsumer cannot connect to zookeeper : Received -1 when reading from channel.

来源：https://stackoverflow.com/questions/30932568/spark-streaming-kafka-integration-direct-approach-eofexception

标签

apache-spark

apache-kafka

streaming

eofexception

spark-streaming-kafka