kafka学习笔记 --- Scala实现Kafka producer 和 consumer

共两种实现方式：

One:

package zetdata

import java.util.Properties

import org.apache.kafka.clients.producer.{KafkaProducer, ProducerRecord}

/**
  * Created by ryan on 17-7-6.
  */
object ScalaProducerExample extends  App{
  val topic = "app"
  val brokers = "192.168.1.81:6667"
  val props = new Properties()
  props.put("bootstrap", brokers)
  props.put("client.id", "ScalaProducerExample")
  props.put("key.serializer",org.apache.kafka.common.serialization.StringSerializer)
  props.put("value.serializer", org.apache.kafka.common.serialization.StringDeserializer)

  val producer = new KafkaProducer[String, String](props)
  val t = System.currentTimeMillis()
  val msg = "hello, I'm test message!"
  val record = new ProducerRecord[String, String](topic, "key", msg)
  producer.send(record)
  producer.close()
}

package zetdata

import java.util.Collections
import java.util.Properties

import org.apache.kafka.clients.consumer.KafkaConsumer

/**
  * Created by ryan on 17-7-6.
  */

object ScalaComsumerExample extends App {
  val topic = "app"
  val brokers = "192.168.1.81:6667"
  val props = new Properties()
  props.put("bootstrap", brokers)
  props.put("client.id", "ScalaProducerExample")
  props.put("key.serializer", org.apache.kafka.common.serialization.StringSerializer)
  props.put("value.serializer", org.apache.kafka.common.serialization.StringDeserializer)
  val consumer = new KafkaConsumer[String, String](props)
  consumer.subscribe(Collections.singleton(topic))
  val records = consumer.poll(1000)
  for (record <- records){
    println(record)
  }
  consumer.close()
}

Two:

package zetdata

import java.util.Properties

import org.apache.kafka.clients.producer._

/**
  * Created by ryan on 17-7-5.
  */
object ProduceExample extends App{
  val props = new Properties()
  val TOPIC = "yangxin_test1"
  props.put("bootstrap.servers", "192.168.1.81:6667")
  props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer")
  props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer")
  props.put("metadata.broker.list", "192.168.1.84:6667")
  props.put("group.id", "something")
  val producer = new KafkaProducer[String, String](props)
  for ( i <- 1 to 50){
      val record = new ProducerRecord(TOPIC, "key", s"hello $i")
      println(record)
      producer.send(record)
  }
  val record = new ProducerRecord(TOPIC, "key", "the end " + new java.util.Date)
  producer.send(record)
  println(record)
  producer.close()
}

package zetdata

import java.util
import java.util.Properties

import scala.collection.JavaConverters._
import org.apache.kafka.clients.consumer.KafkaConsumer

/**
  * Created by ryan on 17-7-5.
  */
object ConsumerExample extends App{
  val TOPIC = "yangxin_test1"
  val props = new Properties()
  props.put("bootstrap.servers", "192.168.1.81:6667")
  props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer")
  props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer")
  props.put("group.id", "something")
  val consumer = new KafkaConsumer[String, String](props)
  consumer.subscribe(util.Collections.singletonList(TOPIC))
  while(true) {
    val records = consumer.poll(100)
    for (record <- records.asScala) {
      println(record)
    }
  }
}

具体运行可在编辑器中

分别运行 producer 与consumer。

或者在linux环境，

利用mavern 

mvn package将程序打包然后：

java -cp kafka_example-0.1.0-SNAPSHOT.jar zetdata.ScalaProducerExample 10000 test_topic localhost:6667

来源：CSDN

作者：杨鑫newlfe

链接：https://blog.csdn.net/u012965373/article/details/74548388