How to get kafka consumer-id for logging

时光毁灭记忆、已成空白 提交于 2019-12-11 10:49:05

问题


In my application i'm using spring-kafka to consume message from kafka server, but from console consumer i get consumer-id of all consumer threads that are active

TOPIC            PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID                                    HOST            CLIENT-ID
easytest-events    9          247367          247367          0             p3-S14-0-e6a1d3cb-8ab3-435f-9f53-5081a6e8f812 /10.66.56.129   p3-S14-0

Is there a way to get consumer-id through code so that i can compare them


回答1:


The consumer-id appears to be the client-id appended with a UUID - so you can just use the client-id (which you can set to whatever you want). Spring will add -0, -1, etc.

You can see the number of threads in the logs as partitions are assigned...

2018-08-31 09:34:27.869  INFO 55748 --- [o52105744-0-C-1] o.s.k.l.KafkaMessageListenerContainer    : partitions assigned: [so52105744-0]
2018-08-31 09:34:27.876  INFO 55748 --- [o52105744-2-C-1] o.s.k.l.KafkaMessageListenerContainer    : partitions assigned: [so52105744-3]
2018-08-31 09:34:27.876  INFO 55748 --- [o52105744-1-C-1] o.s.k.l.KafkaMessageListenerContainer    : partitions assigned: [so52105744-2]
2018-08-31 09:34:27.876  INFO 55748 --- [o52105744-9-C-1] o.s.k.l.KafkaMessageListenerContainer    : partitions assigned: [so52105744-1]
2018-08-31 09:34:27.876  INFO 55748 --- [o52105744-3-C-1] o.s.k.l.KafkaMessageListenerContainer    : partitions assigned: [so52105744-4]
2018-08-31 09:34:27.876  INFO 55748 --- [o52105744-6-C-1] o.s.k.l.KafkaMessageListenerContainer    : partitions assigned: [so52105744-7]
2018-08-31 09:34:27.876  INFO 55748 --- [o52105744-5-C-1] o.s.k.l.KafkaMessageListenerContainer    : partitions assigned: [so52105744-6]
2018-08-31 09:34:27.876  INFO 55748 --- [o52105744-4-C-1] o.s.k.l.KafkaMessageListenerContainer    : partitions assigned: [so52105744-5]
2018-08-31 09:34:27.877  INFO 55748 --- [o52105744-7-C-1] o.s.k.l.KafkaMessageListenerContainer    : partitions assigned: [so52105744-8]
2018-08-31 09:34:27.877  INFO 55748 --- [o52105744-8-C-1] o.s.k.l.KafkaMessageListenerContainer    : partitions assigned: [so52105744-9]



回答2:


I also had the similar use case where I wanted to get the consumer Id for logging so I used the current thread name and it seems to work fine.

LOGGER.info("Current thread: {}",Thread.currentThread().getName());

and this prints:-

Current thread: org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1
Current thread: org.springframework.kafka.KafkaListenerEndpointContainer#0-2-C-1
Current thread: org.springframework.kafka.KafkaListenerEndpointContainer#0-1-C-1

this also shows clearly that the messages are being consumed each time by different thread(may be in the round-robin fashion)



来源:https://stackoverflow.com/questions/52105744/how-to-get-kafka-consumer-id-for-logging

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!