问题
We have java app which runs on app server.
We have topic topic1 with 5 partitions. We have created 5 consumer threads in a consumer group "consumergroup1". One consumer thread per partition. They are running on appserver server one.
Now we bring up one more app server servertwo on another node.
We want to run some of consumers in same group "consumergroup1" for topic1 on new server. Basically distribute consumer threads between two app servers for same consumer group.
If we bring up consumers on server two for consumergroup1 they would be redundant as already 5 consumer threads are running on server one which is equal to total partitions in topic1
Can anyone suggest how to redistribute consumer threads across multiple appservers nodes. The app server nodes are scaled based on demand
来源:https://stackoverflow.com/questions/62662476/scale-kafka-consumers-and-distribute-them-across-multiple-nodes-of-app-server