apache-kafka-streams

问题 I have written this code in a Kafka stream application: KGroupedStream<String, foo> groupedStream = stream.groupByKey(); groupedStream.windowedBy( SessionWindows.with(Duration.ofSeconds(3)).grace(Duration.ofSeconds(3))) .aggregate(() -> {...}) .suppress(Suppressed.untilWindowCloses(unbounded())) .toStream()... which should (if i understood it correctly) emit records per Key after the window is closed. Somehow the behavior is the following: The stream doesn't emit the first record and only

问题 Is there functionality built into Kafka Streams that allows for dynamically connecting a single input stream into multiple output streams? KStream.branch allows branching based on true/false predicates, but this isn't quite what I want. I'd like each incoming log to determine the topic it will be streamed to at runtime, e.g., a log {"date": "2017-01-01"} will be streamed to the topic topic-2017-01-01 and a log {"date": "2017-01-02"} will be streamed to the topic topic-2017-01-02 . I could

问题 In the Kafka Streams Developer Guide it says: Kafka Streams applications can only communicate with a single Kafka cluster specified by this config value. Future versions of Kafka Streams will support connecting to different Kafka clusters for reading input streams and writing output streams. Does this mean that my whole application can only connect to a single Kafka Cluster or each instance of KafkaStreams can only connect to a single cluster? Could I create multiple KafkaStreams instances

问题 I have a basic stream processing flow which looks like master topic -> my processing in a mapper/filter -> output topics and I am wondering about the best way to handle "bad messages". This could potentially be things like messages that I can't deserialize properly, or perhaps the processing/filtering logic fails in some unexpected way (I have no external dependencies so there should be no transient errors of that sort). I was considering wrapping all my processing/filtering code in a try

问题 Is there a way to commit manually with Kafka Stream? Usually with using the KafkaConsumer , I do something like below: while (true) { ConsumerRecords<String, String> records = consumer.poll(100); for (ConsumerRecord<String, String> record : records){ // process records } consumer.commitAsync(); } Where I\'m calling commit manually. I don\'t see a similar API for KStream . 回答1: Commits are handled by Streams internally and fully automatic, and thus there is usually no reason to commit manually