apache-kafka-streams

Say you have a topic with a null key and the value is {id:1, name:Chris, age:99} Lets say you want to count up the number of people by name. You would do something like below: nameStream.groupBy((key,value) -> value.getName()) .count(); Now lets says it is valid you can get duplicate records and you can tell it is a duplicate based on the id. For example: {id:1, name:Chris, age:99} {id:1, name:Chris, age:xx} Should result in a count of one and {id:1, name:Chris, age:99} {id:2, name:Chris, age:xx} should result in a count of 2. How would you accomplish this? I thought reduce would work, but

I am new to developing kafka-streams applications. My stream processor is meant to sort json messages based on a value of a user key in the input json message. Message 1: {"UserID": "1", "Score":"123", "meta":"qwert"} Message 2: {"UserID": "5", "Score":"780", "meta":"mnbvs"} Message 3: {"UserID": "2", "Score":"0", "meta":"fghjk"} I have read here Dynamically connecting a Kafka input stream to multiple output streams that there is no dynamic solution. In my use-case I know the user keys and output topics that I need to sort the input stream. So I am writing separate processor applications

I have a single node kafka broker and simple streams application. I created 2 topics (topic1 and topic2). Produced on topic1 - processed message - write to topic2 Note: For each message produced only one message is written to destination topic I produced a single message. After it was written to topic2, I stopped the kafka broker. After sometime I restarted the broker and produced another message on topic1. Now streams app processed that message 3 times. Now without stopping the broker I produced messages to topic1 and waited for streams app to write to topic2 before producing again. Streams

I'm working with Kafka that save the data into rocksdb. Now I want to have a look at the db keys and values that created by Kafka. I downloaded FastNoSQL and tried but failed. The folder contains: .sst files .log files CURRENT file IDENTITY file LOCK file LOG files MANIFEST files OPTIONS files How can I watch the values? Keylord (since version 5.0) can open RocksDB databases. For example here is Kafka stream of Wordcount application : For RocksDB db files you can use FastoNoSql . It looks like the reason you have data stored in RocksDB files is because you are using Apache Kafka's Streams API.

I wanted to know why does co-partitioning of two Kstreams in kafka require same number of partitions for both the streams as is given in the documentation in below URL: enter link description here As the name "co-partition" indicates, you want to put data from different topic but same key to the same Kafka Streams application instance. If you don't have the same number of partitions, it's not possible to get this behavior. Assume you have topic A with 2 partitions and topic B with 3 partitions. Thus, it can happen that one record with key X is hashed to partitions A-0 and B-1 (ie, not same