how to set group name when consuming messages in kafka using command line

问题

Any idea how to set group name when consuming messages in kafka using command line.

I tried with the following command :

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic nil_RF2_P2 --from-beginning --config group.id=test1
'config' is not a recognized option

The goal is to find the offset of consumed messages with the following command:

bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --zookeeper localhost:2181 --group test1

Can somebody help in this regards!!

Thanks in advance !!

回答1:

The simplest solution is:

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic nil_RF2_P2 --from-beginning --consumer-property group.id=test1

In case you specify the flag --from-beginning just remember that the consumer group should not have consumed any records in the past or else your consumer will start consuming from the earliest not consumed record by the specified group (and not from the actual beginning, as you may incorrectly assume).

回答2:

Got the answer to change the groupname from command prompt!!

steps:

1. create a new consumer.properties file, say consumer1.properties.
2. change group.id=<Give a new group name> in the consumer1.properties.
3. bin/kafka-console-consumer.sh --new-consumer --bootstrap-server localhost:9092 --topic topicname --from-beginning --consumer.config config/consumer1.properties --delete-consumer-offsets

回答3:

if you want change group id without lost offset of record you have get offset manually of current Group.id and set to new run consumer that have new id. if don't have any control to get offset in consumer instance you can run this command.

/bin/kafka-consumer-groups.sh --new-consumer --bootstrap-server <ip_address>:<Broker_port>  --group Group_name --describe

and then you can seek data from specific offset. pay attention you should call seek after call poll، Assign command not work. also you can see my sample of code in github

Example here

回答4:

You can use the --group option like this (tested with Kafka 2.0.0):

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --group test-consumer --topic test --from-beginning

回答5:

If you are using bash then you can use its process substitution capability.

bin/kafka-console-consumer.sh --zookeeper localhost:2181 \
--topic nil_RF2_P2 --from-beginning \
--consumer.config <(echo group.id=test1)

来源：https://stackoverflow.com/questions/38549867/how-to-set-group-name-when-consuming-messages-in-kafka-using-command-line