问题
My project is looking at Apache Kafka as a potential replacement for an aging JMS-based messaging approach. In order to make this transition as smooth as possible, it would be ideal if the replacement queuing system (Kafka) had an asynchronous subscription mechanism, similar to our current project's JMS mechanism of using MessageListener and MessageConsumer to subscribe to topics and receive asynchronous notifications. I don't care so much if Kafka doesn't strictly conform to the JMS API, but conversely, I would prefer not to redesign our entire suite of publish-subscribe-notification classes if I don't need to.
I can find all kinds of KafkaConsumer polling examples, but so far have not been able to find any examples with a client being notified of new messages via asynchronous notification.
Does anyone know if the current version of Kafka (0.10.2 as of the time of this post) provides such an API, or am I stuck with trying to rewrite my legacy code using polling?
回答1:
Kafka clients provides only on-demand pooling mechanism but you can use spring-kafka. It provides MessageListener interface and KafkaListener annotation and similar. See documentation.
回答2:
You may want to try out the Confluent Kafka JMS Client.
http://docs.confluent.io/3.2.0/clients/kafka-jms-client/docs/index.html
Kafka JMS Client is a JMS API wrapper on top of Kafka Producer/Consumer API so it has all the standard JMS 1.1 interfaces. It is an Enterprise (subscription) feature but if you download Confluent Enterprise you can try it out for 30 days free of charge.
https://www.confluent.io/download/
回答3:
In case you can accept a bit of latency after having consumed all the available messages, you can use a timer and call consumer.poll(0), which returns immediately with the available messages. After having consumed the messages you set the timer again with the same acceptable delay, say 100ms.
When throughput is low the batches will be small and this delay will intervene more often. However, since the scenario is asynchronous the delay is not very critical either. You never know exactly when a message arrives anyway.
When the throughput is very high the batches will be large. For the new Kafka consumer the default value of fetch.max.bytes is 52428800. The extra delay will be relatively small compared to the time it takes to process a large batch of messages.
You can wrap this in a small component to which you give the function that corresponds to your current MBean handler.
来源：https://stackoverflow.com/questions/43546252/does-apache-kafka-provide-an-asynchronous-subscription-callback-api